brian/wanderer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into feat/pwa-favicon-support

Browse Source
This commit is contained in:
Brian Nelson
2026-06-23 11:24:03 -04:00
committed by GitHub
parent b96ab15b8c 485ec53f6d
commit cc27a4386c
289 changed files with 25749 additions and 6861 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
.github/workflows/go.yml vendored
View File

@@ -6,19 +6,30 @@ on:
paths:
- '.github/**'
- 'db/**'
- 'plugins/**'
- 'Makefile'
pull_request:
paths:
- '.github/**'
- 'db/**'
- 'plugins/**'
- 'Makefile'
jobs:
db-test:
runs-on: ubuntu-latest
env:
TINYGO_VERSION: '0.39.0'
steps:
- uses: actions/checkout@v6
- uses: actions/setup-go@v6
with:
go-version: '1.25'
- name: Install TinyGo
run: |
curl -fsSL "https://github.com/tinygo-org/tinygo/releases/download/v${TINYGO_VERSION}/tinygo_${TINYGO_VERSION}_amd64.deb" -o /tmp/tinygo.deb
sudo dpkg -i /tmp/tinygo.deb
tinygo version
- run: make db-fmt
- name: Ensure formatting
@@ -31,3 +42,5 @@ jobs:
working-directory: db
- run: make db-vet
- run: make db-test
- run: make plugins-test
- run: make plugins-build

23
.github/workflows/release.yaml vendored
View File

@@ -41,7 +41,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@v6
with:
go-version: '1.22'
go-version: '1.25'
- name: Setup Node.js
uses: actions/setup-node@v6
@@ -74,9 +74,25 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: write
env:
TINYGO_VERSION: '0.39.0'
steps:
- uses: actions/checkout@v6
- name: Setup Go
uses: actions/setup-go@v6
with:
go-version: '1.25'
- name: Install TinyGo
run: |
curl -fsSL "https://github.com/tinygo-org/tinygo/releases/download/v${TINYGO_VERSION}/tinygo_${TINYGO_VERSION}_amd64.deb" -o /tmp/tinygo.deb
sudo dpkg -i /tmp/tinygo.deb
tinygo version
- name: Build Plugin Release Assets
run: make plugins-package
- name: Extract release notes
id: changelog
run: |
@@ -89,9 +105,12 @@ jobs:
echo 'EOF' >> $GITHUB_OUTPUT
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
tag_name: ${{ needs.publish.outputs.version }}
body: ${{ steps.changelog.outputs.changelog }}
files: |
plugin_dist/*.tar.gz
plugin_dist/SHA256SUMS
draft: false
prerelease: false

7
.gitignore vendored
View File

@@ -12,3 +12,10 @@ build*.sh
start*.*
data*/
plugins/*/dist/
plugin_dist/
.planning/
.claude/
CLAUDE.md

17
CHANGELOG.md
View File

@@ -1,3 +1,19 @@
# [Unreleased]
## Features
- Server-side map clustering and zoom-aware polyline filtering: The world map now performs trail clustering on the server to improve performance. At lower zoom levels, smaller trails are clustered, while at higher zoom levels the largest routes in the current view are shown as detailed polylines. The maximum number of simultaneously visible polylines can be configured via the PUBLIC_MAP_MAX_POLYLINES environment variable.
# v0.19.2
## Documentation
- Add CONTRIBUTING guidelines
## Bug Fixes
- All photos from strava activities are now synced, instead of just the first one
- Shared trails are now displayed correctly in search results
- Fixes bug that caused trails to be indexed multiple times causing high server load
- Remaining likes are no correctly calculated when unliking a trail
- Fix waypoint creation from photos
# v0.19.1
## Features
@@ -64,7 +80,6 @@
## Maintenance
- Meilisearch, PocketBase, Go, web/docs dependencies, CI actions, and Docker build setup updated.
# v0.18.5
## Security
- Fixes CVE-2022-39299 via xmldom upgrade (PR #820)

97
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,97 @@
# Contributing
Thank you for your interest in contributing to `wanderer`. We appreciate all contributions, whether they are bug fixes, documentation updates, translations, or new features.
## Before you start
Before working on a significant change, please:
- Open an issue,
- start a discussion
- or contact us via our [Discord server](https://discord.gg/USSEBY98CP) on the `#dev` channel.
This helps us avoid duplicate work and ensures that your changes align with the project's direction.
For small bug or typo fixes, documentation updates, translations, or clearly isolated improvements, opening a pull request directly is usually fine.
Please follow our [local development guide](https://wanderer.to/develop/local-development/).
## Pull Request Target Branch
Please open pull requests only against the `main` branch.
Pull requests targeting release branches, development branches, or unrelated branches may be closed without review.
## Keep pull requests atomic
Please keep pull requests as small and focused as possible.
A good pull request should address one specific topic, such as:
- One bug fix
- One isolated feature
- One documentation improvement
- One dependency update
- One refactoring
Please avoid combining unrelated changes in the same pull request. For example, do not combine a bug fix with formatting changes, dependency updates, or other refactorings.
Smaller pull requests are easier to review, test, and merge.
## Describe the change clearly
Every pull request should include a clear description of the change.
- What was changed
- Why the change was needed
- How the change was tested
- Any known limitations or side effects.
For UI changes, please include screenshots when helpful.
## Bug fixes and reproduction steps
When fixing a bug, please describe how it can be reproduced from an end-user perspective.
Useful reproduction steps explain the actions a user takes in the application and the problem they observe.
Avoid relying solely on artificial or highly technical steps, such as direct API calls to internal endpoints, unless the issue is specifically related to the API or cannot be reasonably reproduced through the user interface.
## AI-assisted contributions
Using AI tools for assistance is allowed. While AI tools can be helpful, contributors are expected to treat AI-assisted changes like any other code they submit. They should understand, carefully review, and test the changes before opening a pull request.
If you used AI to implement a feature, improve existing functionality, or fix an issue, your pull request should clearly explain the problem or improvement, the intended user-facing behavior, and how the change was tested.
To keep review work manageable, we may close pull requests that appear to be mainly AI-generated or submitted in large numbers without clear evidence that the contributor has reviewed, tested, and understood the changes.
## Testing
Please test your changes before opening a pull request.
If automated tests exist for the affected area, please run them. If no automated tests exist, describe the manual testing you performed.
A useful test description can include the following:
- Operating system/browser/environment
- Relevant configuration
- Exact steps tested
- Expected and actual results
## Breaking changes and migrations
If your PR introduces breaking changes or requires migration, clearly state this in the pull request description.
## Security Issues
Please do not report security vulnerabilities through public issues or pull requests.
If you believe you have found one, please contact the maintainers privately first. You can reach us via our [Discord server](https://discord.gg/USSEBY98CP).
## Reviews
Maintainers may request changes, additional tests, a smaller scope, or a different implementation approach.
Please keep discussions constructive and focused on the code and the user-facing behavior.
Thank you for helping improve the project.

32
Makefile
View File

@@ -41,3 +41,35 @@ web-test:
.PHONY: web-build-docker
web-build-docker:
docker buildx build web/ --no-cache -t flomp/wanderer-web:latest
## Plugins
.PHONY: plugins-test
plugins-test:
cd plugins/sdk && go test ./...
cd plugins/hammerhead && go test ./...
cd plugins/komoot && go test ./...
cd plugins/strava && go test ./...
.PHONY: plugins-build
plugins-build:
cd plugins/hammerhead && XDG_CACHE_HOME=$${XDG_CACHE_HOME:-/tmp/wanderer-tinygo-cache} make build
cd plugins/komoot && XDG_CACHE_HOME=$${XDG_CACHE_HOME:-/tmp/wanderer-tinygo-cache} make build
cd plugins/strava && XDG_CACHE_HOME=$${XDG_CACHE_HOME:-/tmp/wanderer-tinygo-cache} make build
.PHONY: plugins-install-local
plugins-install-local: plugins-build
mkdir -p data/plugins
rm -rf data/plugins/hammerhead data/plugins/komoot data/plugins/strava
cp -a plugins/hammerhead/dist/hammerhead data/plugins/
cp -a plugins/komoot/dist/komoot data/plugins/
cp -a plugins/strava/dist/strava data/plugins/
.PHONY: plugins-package
plugins-package: plugins-build
rm -rf plugin_dist
mkdir -p plugin_dist
tar -C plugins/hammerhead/dist -czf plugin_dist/wanderer-plugin-hammerhead.tar.gz hammerhead
tar -C plugins/komoot/dist -czf plugin_dist/wanderer-plugin-komoot.tar.gz komoot
tar -C plugins/strava/dist -czf plugin_dist/wanderer-plugin-strava.tar.gz strava
cd plugin_dist && sha256sum *.tar.gz > SHA256SUMS

2
db/.dockerignore
View File

@@ -6,6 +6,8 @@
!integrations
!main.go
!migrations
!plugins
!pluginsystem
!routes
!templates
!services

32
db/federation/activity.go
View File

@@ -18,11 +18,40 @@ import (
"github.com/go-ap/jsonld"
"github.com/go-fed/httpsig"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
"golang.org/x/sync/semaphore"
)
var httpClient = &http.Client{Timeout: 10 * time.Second}
// followerInboxes returns inbox URLs for all accepted followers of actorId
// in a single JOIN query instead of one query per follower.
func followerInboxes(app core.App, actorId string) ([]string, error) {
rows, err := app.DB().
Select("aa.inbox").
From("follows f").
InnerJoin("activitypub_actors aa", dbx.NewExp("f.follower = aa.id")).
Where(dbx.NewExp("f.followee = {:followee} AND f.status = 'accepted' AND aa.inbox != ''",
dbx.Params{"followee": actorId})).
Rows()
if err != nil {
return nil, err
}
defer rows.Close()
var inboxes []string
for rows.Next() {
var inbox string
if err := rows.Scan(&inbox); err != nil {
return nil, err
}
inboxes = append(inboxes, inbox)
}
return inboxes, rows.Err()
}
func PostActivity(app core.App, actor *core.Record, activity *pub.Activity, recipients []string) error {
go func() {
defer func() {
@@ -67,7 +96,6 @@ func PostActivity(app core.App, actor *core.Record, activity *pub.Activity, reci
}
pubID := actor.GetString("iri") + "#main-key"
client := &http.Client{}
sem := semaphore.NewWeighted(5)
slices.Sort(recipients)
@@ -105,7 +133,7 @@ func PostActivity(app core.App, actor *core.Record, activity *pub.Activity, reci
return
}
resp, err := client.Do(req)
resp, err := httpClient.Do(req)
if err != nil {
app.Logger().Error(fmt.Sprintf("Error sending to inbox %s: %s", inbox, err))
return

37
db/federation/actor.go
View File

@@ -69,7 +69,7 @@ func GetActorByHandle(app core.App, ctx context.Context, handle string, includeF
if domain != "" {
filter += "domain={:domain}"
} else {
filter += "isLocal=true"
filter += "is_local=true"
}
var dbActor *core.Record
@@ -81,7 +81,7 @@ func GetActorByHandle(app core.App, ctx context.Context, handle string, includeF
}
dbActor = core.NewRecord(collection)
dbActor.Set("isLocal", false)
dbActor.Set("is_local", false)
iri, err := iriFromHandle(ctx, domain, username)
if err != nil {
return nil, err
@@ -105,7 +105,7 @@ func GetActorByIRI(app core.App, ctx context.Context, iri string, includeFollows
}
dbActor = core.NewRecord(collection)
dbActor.Set("isLocal", false)
dbActor.Set("is_local", false)
dbActor.Set("iri", iri)
} else if err != nil {
@@ -167,7 +167,7 @@ func assembleActor(app core.App, ctx context.Context, dbActor *core.Record, incl
}
private := false
if dbActor.GetBool("isLocal") {
if dbActor.GetBool("is_local") {
user, err := app.FindRecordById("users", dbActor.GetString("user"))
if err != nil {
return nil, err
@@ -194,12 +194,21 @@ func assembleActor(app core.App, ctx context.Context, dbActor *core.Record, incl
dbActor.Set("last_fetched", time.Now())
// an empty privacy field is the default for users who never touched
// their privacy settings and is treated as public. A non-empty but
// corrupt value fails closed (private) so a broken setting can't
// silently expose a profile.
privacy := settings.GetString("privacy")
if privacy != "" {
result := make(map[string]interface{})
json.Unmarshal([]byte(privacy), &result)
if err := json.Unmarshal([]byte(privacy), &result); err != nil {
private = true
} else {
// check that it's not our own profile
private = result["account"] == "private" && dbActor.Id != strings.TrimPrefix(ctx.Value("actor").(string), "actor:")
actorVal, _ := ctx.Value("actor").(string)
private = result["account"] == "private" && dbActor.Id != strings.TrimPrefix(actorVal, "actor:")
}
}
} else {
@@ -279,6 +288,9 @@ func fetchRemoteActor(app core.App, ctx context.Context, iri string, includeFoll
client := util.SafeHTTPClient()
req, err := http.NewRequestWithContext(ctx, "GET", iri, nil)
if err != nil {
return nil, nil, nil, err
}
headers := map[string]string{
"Accept": "application/ld+json",
@@ -291,7 +303,8 @@ func fetchRemoteActor(app core.App, ctx context.Context, iri string, includeFoll
req.Header.Add(k, v)
}
userActorId := strings.TrimPrefix(ctx.Value("actor").(string), "actor:")
actorVal, _ := ctx.Value("actor").(string)
userActorId := strings.TrimPrefix(actorVal, "actor:")
userActor, err := app.FindRecordById("activitypub_actors", userActorId)
if userActor != nil && userActor.GetString("private_key") != "" {
dbPrivateKey := userActor.GetString("private_key")
@@ -332,7 +345,7 @@ func fetchRemoteActor(app core.App, ctx context.Context, iri string, includeFoll
defer resp.Body.Close()
var pubActor pub.Actor
if err := json.NewDecoder(resp.Body).Decode(&pubActor); err != nil {
if err := json.NewDecoder(io.LimitReader(resp.Body, 1<<20)).Decode(&pubActor); err != nil {
return nil, nil, nil, err
}
@@ -365,6 +378,9 @@ func FetchCollection(app core.App, ctx context.Context, collectionURL string) (*
}
req, err := http.NewRequestWithContext(ctx, "GET", collectionURL, nil)
if err != nil {
return nil, err
}
headers := map[string]string{
"Accept": "application/ld+json",
@@ -376,7 +392,8 @@ func FetchCollection(app core.App, ctx context.Context, collectionURL string) (*
for k, v := range headers {
req.Header.Add(k, v)
}
userActorId := strings.TrimPrefix(ctx.Value("actor").(string), "actor:")
actorVal, _ := ctx.Value("actor").(string)
userActorId := strings.TrimPrefix(actorVal, "actor:")
userActor, err := app.FindRecordById("activitypub_actors", userActorId)
if userActor != nil && userActor.GetString("private_key") != "" {
dbPrivateKey := userActor.GetString("private_key")

26
db/federation/announce.go
View File

@@ -3,9 +3,7 @@ package federation
import (
"database/sql"
"fmt"
"net/url"
"os"
"path"
"pocketbase/util"
"strings"
"time"
@@ -115,14 +113,12 @@ func ProcessAnnounceActivity(app core.App, actor *core.Record, activity pub.Acti
object := activity.Object.GetID().String()
if strings.Contains(object, "/api/v1/trail") {
processTrailAnnounceActivity(app, actor, activity)
return processTrailAnnounceActivity(app, actor, activity)
} else if strings.Contains(object, "/api/v1/list") {
processListAnnounceActivity(app, actor, activity)
return processListAnnounceActivity(app, actor, activity)
} else {
return fmt.Errorf("unknown announce type")
}
return nil
}
@@ -134,7 +130,7 @@ func processTrailAnnounceActivity(app core.App, actor *core.Record, activity pub
}
var trail *core.Record
if !actor.GetBool("isLocal") {
if !actor.GetBool("is_local") {
trail, err = util.TrailFromActivity(activity, app, actor)
if err != nil {
return err
@@ -180,12 +176,7 @@ func processTrailAnnounceActivity(app core.App, actor *core.Record, activity pub
return err
}
} else {
trailUrl, err := url.Parse(activity.Object.GetID().String())
if err != nil {
return err
}
trailId := path.Base(trailUrl.Path)
trail, err = app.FindRecordById("trails", trailId)
trail, err = app.FindFirstRecordByData("trails", "iri", activity.Object.GetID().String())
if err != nil {
return err
}
@@ -217,7 +208,7 @@ func processListAnnounceActivity(app core.App, actor *core.Record, activity pub.
}
var list *core.Record
if !actor.GetBool("isLocal") {
if !actor.GetBool("is_local") {
list, err = util.ListFromActivity(activity, app, actor)
if err != nil {
return err
@@ -245,12 +236,7 @@ func processListAnnounceActivity(app core.App, actor *core.Record, activity pub.
return err
}
} else {
listUrl, err := url.Parse(activity.Object.GetID().String())
if err != nil {
return err
}
listId := path.Base(listUrl.Path)
list, err = app.FindRecordById("trails", listId)
list, err = app.FindFirstRecordByData("lists", "iri", activity.Object.GetID().String())
if err != nil {
return err
}

93
db/federation/create.go
View File

@@ -4,9 +4,7 @@ import (
"context"
"database/sql"
"fmt"
"net/url"
"os"
"path"
"strconv"
"strings"
"time"
@@ -14,7 +12,6 @@ import (
"pocketbase/util"
pub "github.com/go-ap/activitypub"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/filesystem"
"github.com/pocketbase/pocketbase/tools/security"
@@ -91,19 +88,11 @@ func CreateTrailActivity(app core.App, ctx context.Context, trail *core.Record,
return err
}
follows, err := app.FindRecordsByFilter("follows", "followee={:followee}&&status='accepted'", "", -1, 0, dbx.Params{"followee": trailAuthor.Id})
inboxes, err := followerInboxes(app, trailAuthor.Id)
if err != nil {
return err
}
recipients := mentions
for _, f := range follows {
follower, err := app.FindRecordById("activitypub_actors", f.GetString("follower"))
if err != nil {
return err
}
recipients = append(recipients, follower.GetString("inbox"))
}
recipients := append(mentions, inboxes...)
return PostActivity(app, trailAuthor, activity, recipients)
}
@@ -228,12 +217,7 @@ func CreateSummitLogActivity(app core.App, ctx context.Context, summitLog *core.
}
var trailIRI pub.IRI
if summitLogTrailAuthor.GetBool("isLocal") {
trailId := summitLog.GetString("trail")
trailIRI = pub.IRI(fmt.Sprintf("%s/api/v1/trail/%s", origin, trailId))
} else {
trailIRI = pub.IRI(summitLogTrail.GetString("iri"))
}
recordId := security.RandomStringWithAlphabet(core.DefaultIdLength, core.DefaultIdAlphabet)
@@ -271,15 +255,14 @@ func CreateSummitLogActivity(app core.App, ctx context.Context, summitLog *core.
gpx = fmt.Sprintf("%s/api/v1/files/summit_logs/%s/%s", origin, summitLog.Id, summitLog.GetString("gpx"))
}
attachments := make(pub.ItemCollection, max(len(photos), 2))
attachments := make(pub.ItemCollection, 0, len(photos)+1)
for i := range len(photos) {
iri := fmt.Sprintf("%s/api/v1/files/summit_logs/%s/%s", origin, summitLog.Id, photos[i])
attachments[i] = pub.Document{
attachments.Append(pub.Document{
Type: pub.ImageType,
MediaType: "image/jpeg",
URL: pub.IRI(iri),
}
})
}
if gpx != "" {
attachments.Append(pub.Document{
@@ -321,7 +304,7 @@ func CreateSummitLogActivity(app core.App, ctx context.Context, summitLog *core.
logObject.Content = pub.NaturalLanguageValuesNew(pub.LangRefValueNew(pub.NilLangRef, summitLog.GetString("text")))
logObject.AttributedTo = pub.IRI(summitLogAuthor.GetString("iri"))
logObject.Published = summitLog.GetDateTime("created").Time()
logObject.ID = pub.IRI(fmt.Sprintf("%s/api/v1/summit-log/%s", origin, summitLog.Id))
logObject.ID = pub.IRI(summitLog.GetString("iri"))
logObject.URL = pub.IRI(fmt.Sprintf("%s/trail/view/@%s/%s", origin, summitLogTrailAuthor.GetString("preferred_username"), summitLog.GetString("trail")))
logObject.InReplyTo = trailIRI
logObject.Tag = tags
@@ -335,20 +318,11 @@ func CreateSummitLogActivity(app core.App, ctx context.Context, summitLog *core.
activity.CC = cc
activity.Published = time.Now()
follows, err := app.FindRecordsByFilter("follows", "followee={:followee}&&status='accepted'", "", -1, 0, dbx.Params{"followee": summitLogAuthor.Id})
inboxes, err := followerInboxes(app, summitLogAuthor.Id)
if err != nil {
return err
}
recipients := mentions
for _, f := range follows {
follower, err := app.FindRecordById("activitypub_actors", f.GetString("follower"))
if err != nil {
return err
}
recipients = append(recipients, follower.GetString("inbox"))
}
recipients := append(mentions, inboxes...)
if summitLogAuthor.Id != summitLogTrailAuthor.Id {
recipients = append(recipients, summitLogTrailAuthor.GetString("inbox"))
@@ -412,20 +386,11 @@ func CreateListActivity(app core.App, list *core.Record, typ pub.ActivityVocabul
return err
}
follows, err := app.FindRecordsByFilter("follows", "followee={:followee}&&status='accepted'", "", -1, 0, dbx.Params{"followee": listAuthor.Id})
recipients, err := followerInboxes(app, listAuthor.Id)
if err != nil {
return err
}
recipients := []string{}
for _, f := range follows {
follower, err := app.FindRecordById("activitypub_actors", f.GetString("follower"))
if err != nil {
return err
}
recipients = append(recipients, follower.GetString("inbox"))
}
err = PostActivity(app, listAuthor, activity, recipients)
if err != nil {
return err
@@ -476,7 +441,10 @@ func processCreateOrUpdateTrailActivity(activity pub.Activity, app core.App, act
return err
}
trailObject, _ := pub.ToObject(activity.Object)
trailObject, err := pub.ToObject(activity.Object)
if err != nil {
return err
}
for _, t := range trailObject.Tag {
if t.GetType() == pub.MentionType {
@@ -494,11 +462,11 @@ func processCreateOrUpdateTrailActivity(activity pub.Activity, app core.App, act
Seen: false,
Author: actor.Id,
}
return util.SendNotification(app, notification, mentionedActor)
util.SendNotification(app, notification, mentionedActor)
}
}
return err
return nil
}
func processCreateOrUpdateCommentActivity(activity pub.Activity, app core.App, actor *core.Record) error {
@@ -512,14 +480,8 @@ func processCreateOrUpdateCommentActivity(activity pub.Activity, app core.App, a
return fmt.Errorf("error processing comment: InReplyTo empty")
}
trailUrl, err := url.Parse(commentObject.InReplyTo.GetLink().String())
if err != nil {
return err
}
trailId := path.Base(trailUrl.Path)
var trail *core.Record
trail, err = app.FindFirstRecordByFilter("trails", "iri={:iri} || id={:id}", dbx.Params{"id": trailId, "iri": commentObject.InReplyTo.GetID().String()})
trail, err = app.FindFirstRecordByData("trails", "iri", commentObject.InReplyTo.GetLink().String())
// if the trail is not present on this instance fetch it
if err != nil {
@@ -544,7 +506,7 @@ func processCreateOrUpdateCommentActivity(activity pub.Activity, app core.App, a
}
// no need to do anything else if the actor is local
if actor.GetBool("isLocal") {
if actor.GetBool("is_local") {
return nil
}
@@ -591,7 +553,7 @@ func processCreateOrUpdateCommentActivity(activity pub.Activity, app core.App, a
Seen: false,
Author: actor.Id,
}
return util.SendNotification(app, notification, mentionedActor)
util.SendNotification(app, notification, mentionedActor)
}
}
if activity.Type == pub.CreateType {
@@ -619,13 +581,7 @@ func processCreateOrUpdateSummitLogActivity(activity pub.Activity, app core.App,
return err
}
trailIRI, err := url.Parse(logObject.InReplyTo.GetID().String())
if err != nil {
return err
}
trailId := path.Base(trailIRI.Path)
trail, err := app.FindFirstRecordByFilter("trails", "iri={:iri} || id={:id}", dbx.Params{"id": trailId, "iri": logObject.InReplyTo.GetID().String()})
trail, err := app.FindFirstRecordByData("trails", "iri", logObject.InReplyTo.GetID().String())
// if the trail is not present on this instance fetch it
if err != nil {
if err == sql.ErrNoRows {
@@ -648,6 +604,11 @@ func processCreateOrUpdateSummitLogActivity(activity pub.Activity, app core.App,
return err
}
// no need to do anything else if the actor is local
if actor.GetBool("is_local") {
return nil
}
newSummitLog := false
record, err := app.FindFirstRecordByData("summit_logs", "iri", logObject.ID.String())
if err != nil {
@@ -663,10 +624,6 @@ func processCreateOrUpdateSummitLogActivity(activity pub.Activity, app core.App,
return err
}
}
// no need to do anything else if the actor is local
if actor.GetBool("isLocal") {
return nil
}
var distance, duration, elevation_gain, elevation_loss float64
tags, err := pub.ToItemCollection(logObject.Tag)
@@ -771,7 +728,7 @@ func processCreateOrUpdateSummitLogActivity(activity pub.Activity, app core.App,
Seen: false,
Author: actor.Id,
}
return util.SendNotification(app, notification, mentionedActor)
util.SendNotification(app, notification, mentionedActor)
}
}

57
db/federation/delete.go
View File

@@ -9,7 +9,6 @@ import (
pub "github.com/go-ap/activitypub"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
)
@@ -29,6 +28,10 @@ func CreateTrailDeleteActivity(app core.App, r *core.Record) error {
return err
}
if !author.GetBool("is_local") {
return nil
}
collection, err := app.FindCollectionByNameOrId("activitypub_activities")
if err != nil {
return err
@@ -39,7 +42,7 @@ func CreateTrailDeleteActivity(app core.App, r *core.Record) error {
id := fmt.Sprintf("%s/api/v1/activitypub/activity/%s", origin, recordId)
to := "https://www.w3.org/ns/activitystreams#Public"
cc := author.GetString("iri") + "/followers"
object := fmt.Sprintf("%s/api/v1/trail/%s", origin, r.Id)
object := r.GetString("iri")
record := core.NewRecord(collection)
record.Set("id", recordId)
@@ -62,20 +65,11 @@ func CreateTrailDeleteActivity(app core.App, r *core.Record) error {
activity.CC = pub.ItemCollection{pub.IRI(cc)}
activity.Published = time.Now()
follows, err := app.FindRecordsByFilter("follows", "followee={:followee}&&status='accepted'", "", -1, 0, dbx.Params{"followee": author.Id})
recipients, err := followerInboxes(app, author.Id)
if err != nil {
return err
}
recipients := []string{}
for _, f := range follows {
follower, err := app.FindRecordById("activitypub_actors", f.GetString("follower"))
if err != nil {
return err
}
recipients = append(recipients, follower.GetString("inbox"))
}
return PostActivity(app, author, activity, recipients)
}
@@ -91,7 +85,7 @@ func CreateCommentDeleteActivity(app core.App, client meilisearch.ServiceManager
return err
}
if !author.GetBool("isLocal") {
if !author.GetBool("is_local") {
return nil
}
@@ -105,7 +99,7 @@ func CreateCommentDeleteActivity(app core.App, client meilisearch.ServiceManager
return err
}
if commentTrailAuthor.GetBool("isLocal") {
if commentTrailAuthor.GetBool("is_local") {
return nil
}
@@ -118,7 +112,7 @@ func CreateCommentDeleteActivity(app core.App, client meilisearch.ServiceManager
id := fmt.Sprintf("%s/api/v1/activitypub/activity/%s", origin, recordId)
to := commentTrailAuthor.GetString("iri")
object := fmt.Sprintf("%s/api/v1/comment/%s", origin, r.Id)
object := r.GetString("iri")
activity := pub.DeleteNew(pub.IRI(id), pub.IRI(object))
activity.Actor = pub.IRI(author.GetString("iri"))
@@ -153,7 +147,7 @@ func CreateSummitLogDeleteActivity(app core.App, r *core.Record) error {
return err
}
if !author.GetBool("isLocal") {
if !author.GetBool("is_local") {
return nil
}
@@ -176,7 +170,7 @@ func CreateSummitLogDeleteActivity(app core.App, r *core.Record) error {
id := fmt.Sprintf("%s/api/v1/activitypub/activity/%s", origin, recordId)
to := summitLogTrailAuthor.GetString("iri")
object := fmt.Sprintf("%s/api/v1/summit-log/%s", origin, r.Id)
object := r.GetString("iri")
cc := pub.ItemCollection{pub.IRI(author.GetString("iri") + "/followers")}
activity := pub.DeleteNew(pub.IRI(id), pub.IRI(object))
@@ -185,21 +179,11 @@ func CreateSummitLogDeleteActivity(app core.App, r *core.Record) error {
activity.CC = cc
activity.Published = time.Now()
follows, err := app.FindRecordsByFilter("follows", "followee={:followee}&&status='accepted'", "", -1, 0, dbx.Params{"followee": author.Id})
recipients, err := followerInboxes(app, author.Id)
if err != nil {
return err
}
recipients := []string{}
for _, f := range follows {
follower, err := app.FindRecordById("activitypub_actors", f.GetString("follower"))
if err != nil {
return err
}
recipients = append(recipients, follower.GetString("inbox"))
}
if author.Id != summitLogTrailAuthor.Id {
recipients = append(recipients, summitLogTrailAuthor.GetString("inbox"))
}
@@ -234,7 +218,7 @@ func CreateListDeleteActivity(app core.App, r *core.Record) error {
return err
}
if !author.GetBool("isLocal") {
if !author.GetBool("is_local") {
return nil
}
@@ -248,7 +232,7 @@ func CreateListDeleteActivity(app core.App, r *core.Record) error {
id := fmt.Sprintf("%s/api/v1/activitypub/activity/%s", origin, recordId)
to := "https://www.w3.org/ns/activitystreams#Public"
cc := author.GetString("iri") + "/followers"
object := fmt.Sprintf("%s/api/v1/list/%s", origin, r.Id)
object := r.GetString("iri")
activity := pub.DeleteNew(pub.IRI(id), pub.IRI(object))
activity.Actor = pub.IRI(author.GetString("iri"))
@@ -256,20 +240,11 @@ func CreateListDeleteActivity(app core.App, r *core.Record) error {
activity.CC = pub.ItemCollection{pub.IRI(cc)}
activity.Published = time.Now()
follows, err := app.FindRecordsByFilter("follows", "followee={:followee}&&status='accepted'", "", -1, 0, dbx.Params{"followee": author.Id})
recipients, err := followerInboxes(app, author.Id)
if err != nil {
return err
}
recipients := []string{}
for _, f := range follows {
follower, err := app.FindRecordById("activitypub_actors", f.GetString("follower"))
if err != nil {
return err
}
recipients = append(recipients, follower.GetString("inbox"))
}
err = PostActivity(app, author, activity, recipients)
if err != nil {
return err
@@ -290,7 +265,7 @@ func CreateListDeleteActivity(app core.App, r *core.Record) error {
func ProcessDeleteActivity(app core.App, actor *core.Record, activity pub.Activity) error {
// no need to do anything if the actor is local
if actor.GetBool("isLocal") {
if actor.GetBool("is_local") {
return nil
}

2
db/federation/follow.go
View File

@@ -76,7 +76,7 @@ func ProcessFollowActivity(app core.App, actor *core.Record, activity pub.Activi
// a remote actor has requested the follow
// this means we have not yet created a follow entry in our db
// we accept it immediately
if !actor.GetBool("isLocal") {
if !actor.GetBool("is_local") {
followCollection, err := app.FindCollectionByNameOrId("follows")
if err != nil {
return err

11
db/federation/like.go
View File

@@ -3,7 +3,6 @@ package federation
import (
"fmt"
"os"
"path"
"pocketbase/util"
"time"
@@ -36,11 +35,6 @@ func CreateLikeActivity(app core.App, like *core.Record) error {
object := trail.GetString("iri")
if object == "" {
// trail is local
object = fmt.Sprintf("%s/api/v1/trail/%s", origin, trail.Id)
}
collection, err := app.FindCollectionByNameOrId("activitypub_activities")
if err != nil {
return err
@@ -76,8 +70,7 @@ func ProcessLikeActivity(app core.App, actor *core.Record, activity pub.Activity
return fmt.Errorf("ORIGIN not set")
}
trailId := path.Base(activity.Object.GetID().String())
trail, err := app.FindRecordById("trails", trailId)
trail, err := app.FindFirstRecordByData("trails", "iri", activity.Object.GetID().String())
if err != nil {
return err
}
@@ -87,7 +80,7 @@ func ProcessLikeActivity(app core.App, actor *core.Record, activity pub.Activity
return err
}
if !actor.GetBool("isLocal") {
if !actor.GetBool("is_local") {
trailLikeCollection, err := app.FindCollectionByNameOrId("trail_like")
if err != nil {
return err

12
db/federation/undo.go
View File

@@ -3,7 +3,6 @@ package federation
import (
"fmt"
"os"
"path"
"time"
pub "github.com/go-ap/activitypub"
@@ -91,10 +90,6 @@ func CreateUnlikeActivity(app core.App, like *core.Record) error {
}
object := trail.GetString("iri")
if object == "" {
// trail is local
object = fmt.Sprintf("%s/api/v1/trail/%s", origin, trail.Id)
}
// find the original follow activity
likeActivityRecord, err := app.FindFirstRecordByFilter("activitypub_activities", "actor={:actor}&&object={:object}&&type={:type}", dbx.Params{"actor": actor.GetString("iri"), "object": object, "type": string(pub.LikeType)})
@@ -145,7 +140,7 @@ func ProcessUndoActivity(app core.App, actor *core.Record, activity pub.Activity
func processUnfollowActivity(app core.App, actor *core.Record, activity pub.Activity) error {
// this was a local follow
if actor.GetBool("isLocal") {
if actor.GetBool("is_local") {
return nil
}
@@ -169,14 +164,13 @@ func processUnfollowActivity(app core.App, actor *core.Record, activity pub.Acti
}
func processUnlikeActivity(app core.App, actor *core.Record, activity pub.Activity) error {
if actor.GetBool("isLocal") {
if actor.GetBool("is_local") {
return nil
}
likeActivity := activity.Object.(*pub.Activity)
trailId := path.Base(likeActivity.Object.GetID().String())
trail, err := app.FindRecordById("trails", trailId)
trail, err := app.FindFirstRecordByData("trails", "iri", likeActivity.Object.GetID().String())
if err != nil {
return err
}

19
db/go.mod
View File

@@ -3,6 +3,8 @@ module pocketbase
go 1.25.0
require (
github.com/doyensec/safeurl v0.2.3
github.com/extism/go-sdk v1.7.1
github.com/go-ap/jsonld v0.0.0-20250905102310-8480b0fe24d9
github.com/meilisearch/meilisearch-go v0.36.2
github.com/pocketbase/dbx v1.12.0
@@ -13,12 +15,19 @@ require (
require (
git.sr.ht/~mariusor/go-xsd-duration v0.0.0-20220703122237-02e73435a078 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/go-ap/errors v0.0.0-20250905102357-4480b47a00c4 // indirect
github.com/go-sql-driver/mysql v1.9.3 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca // indirect
github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 // indirect
github.com/tetratelabs/wazero v1.9.0 // indirect
github.com/valyala/fastjson v1.6.10 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)
require (
@@ -32,7 +41,7 @@ require (
github.com/ganigeorgiev/fexpr v0.5.0 // indirect
github.com/go-ap/activitypub v0.0.0-20250905102448-e9df599e4528
github.com/go-fed/httpsig v1.1.0
github.com/go-ozzo/ozzo-validation/v4 v4.3.0 // indirect
github.com/go-ozzo/ozzo-validation/v4 v4.3.0
github.com/google/uuid v1.6.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
@@ -44,13 +53,13 @@ require (
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10 // indirect
github.com/twpayne/go-polyline v1.1.1
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/crypto v0.51.0 // indirect
golang.org/x/image v0.39.0 // indirect
golang.org/x/net v0.53.0
golang.org/x/net v0.55.0
golang.org/x/oauth2 v0.36.0 // indirect
golang.org/x/sync v0.20.0
golang.org/x/sys v0.43.0 // indirect
golang.org/x/text v0.36.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/text v0.37.0 // indirect
modernc.org/libc v1.72.0 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect

46
db/go.sum
View File

@@ -17,9 +17,15 @@ github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
github.com/domodwyer/mailyak/v3 v3.6.2 h1:x3tGMsyFhTCaxp6ycgR0FE/bu5QiNp+hetUuCOBXMn8=
github.com/domodwyer/mailyak/v3 v3.6.2/go.mod h1:lOm/u9CyCVWHeaAmHIdF4RiKVxKUT/H5XX10lIKAL6c=
github.com/doyensec/safeurl v0.2.3 h1:KJZHxTUMI17yUSy5umKmDLtzYBUxN6MkdSIyRI81DvY=
github.com/doyensec/safeurl v0.2.3/go.mod h1:3H0cgRpPYPSpgxRRn5yGD35Ns/LgGX/BVWSBbzUqXtY=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/dvyukov/go-fuzz v0.0.0-20200318091601-be3528f3a813/go.mod h1:11Gm+ccJnvAhCNLlf5+cS9KjtbaD5I5zaZpFMsTHWTw=
github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a h1:UwSIFv5g5lIvbGgtf3tVwC7Ky9rmMFBp0RMs+6f6YqE=
github.com/dylibso/observe-sdk/go v0.0.0-20240819160327-2d926c5d788a/go.mod h1:C8DzXehI4zAbrdlbtOByKX6pfivJTBiV9Jjqv56Yd9Q=
github.com/extism/go-sdk v1.7.1 h1:lWJos6uY+tRFdlIHR+SJjwFDApY7OypS/2nMhiVQ9Sw=
github.com/extism/go-sdk v1.7.1/go.mod h1:IT+Xdg5AZM9hVtpFUA+uZCJMge/hbvshl8bwzLtFyKA=
github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
@@ -43,6 +49,8 @@ github.com/go-ozzo/ozzo-validation/v4 v4.3.0/go.mod h1:2NKgrcHl3z6cJs+3Oo940FPRi
github.com/go-sql-driver/mysql v1.4.1/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
github.com/gobwas/glob v0.2.3 h1:A4xDbljILXROh+kObIiy5kIaPYD8e96x1tgBhUI5J+Y=
github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
@@ -56,6 +64,8 @@ github.com/gorilla/css v1.0.1 h1:ntNaBIghp6JmvWnxbZKANoLyuXTPZ4cAMlo6RyhlbO8=
github.com/gorilla/css v1.0.1/go.mod h1:BvnYkspnSzMmwRK+b8/xgNPLiIuNZr6vbZBTPQ2A3b0=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca h1:T54Ema1DU8ngI+aef9ZhAhNGQhcRTrWxVeG07F+c/Rw=
github.com/ianlancetaylor/demangle v0.0.0-20240805132620-81f5be970eca/go.mod h1:gx7rwoVhcfuVKG5uya9Hs3Sxj7EIvldVofAWIUtGouw=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
@@ -97,6 +107,10 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834 h1:ZF+QBjOI+tILZjBaFj3HgFonKXUcwgJ4djLb6i42S3Q=
github.com/tetratelabs/wabin v0.0.0-20230304001439-f6f874872834/go.mod h1:m9ymHTgNSEjuxvw8E7WWe4Pl4hZQHXONY8wE6dMLaRk=
github.com/tetratelabs/wazero v1.9.0 h1:IcZ56OuxrtaEz8UYNRHBrUa9bYeX9oVY93KspZZBf/I=
github.com/tetratelabs/wazero v1.9.0/go.mod h1:TSbcXCfFP0L2FGkRPxHphadXPjo1T6W+CseNNY7EkjM=
github.com/tkrajina/gpxgo v1.4.0 h1:cSD5uSwy3VZuNFieTEZLyRnuIwhonQEkGPkPGW4XNag=
github.com/tkrajina/gpxgo v1.4.0/go.mod h1:BXSMfUAvKiEhMEXAFM2NvNsbjsSvp394mOvdcNjettg=
github.com/twpayne/go-polyline v1.1.1 h1:/tSF1BR7rN4HWj4XKqvRUNrCiYVMCvywxTFVofvDV0w=
@@ -105,21 +119,23 @@ github.com/valyala/fastjson v1.6.10 h1:/yjJg8jaVQdYR3arGxPE2X5z89xrlhS0eGXdv+ADT
github.com/valyala/fastjson v1.6.10/go.mod h1:e6FubmQouUNP73jtMLmcbxS6ydWIpOfhz34TSfO3JaE=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
go.opentelemetry.io/proto/otlp v1.3.1 h1:TrMUixzpM0yuc/znrFTP9MMRh8trP93mkCiDVeXrui0=
go.opentelemetry.io/proto/otlp v1.3.1/go.mod h1:0X1WI4de4ZsLrrJNLAQbFeLCm3T7yBkR0XqQ7niQU+8=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/crypto v0.51.0 h1:IBPXwPfKxY7cWQZ38ZCIRPI50YLeevDLlLnyC5wRGTI=
golang.org/x/crypto v0.51.0/go.mod h1:8AdwkbraGNABw2kOX6YFPs3WM22XqI4EXEd8g+x7Oc8=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/image v0.39.0 h1:skVYidAEVKgn8lZ602XO75asgXBgLj9G/FE3RbuPFww=
golang.org/x/image v0.39.0/go.mod h1:sIbmppfU+xFLPIG0FoVUTvyBMmgng1/XAMhQ2ft0hpA=
golang.org/x/mod v0.34.0 h1:xIHgNUUnW6sYkcM5Jleh05DvLOtwc6RitGHbDk4akRI=
golang.org/x/mod v0.34.0/go.mod h1:ykgH52iCZe79kzLLMhyCUzhMci+nQj+0XkbXpNYtVjY=
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20210614182718-04defd469f4e/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/net v0.55.0 h1:bcvxaJn3e1U6InsFWt1JUq1aSjnRxLzT2rtD2KfkDF8=
golang.org/x/net v0.55.0/go.mod h1:L5U2KuzuOe1lY7Z+aWVIKK6qEeJXnXV9yzGA+WCHJww=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
@@ -128,20 +144,22 @@ golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/sys v0.45.0 h1:dO4czNzziLiiXplLQgBCEpCvXQ3dnkn0SdaZSYdQ+FY=
golang.org/x/sys v0.45.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
golang.org/x/term v0.43.0 h1:S4RLU2sB31O/NCl+zFN9Aru9A/Cq2aqKpTZJ6B+DwT4=
golang.org/x/term v0.43.0/go.mod h1:lrhlHNdQJHO+1qVYiHfFKVuVioJIheAc3fBSMFYEIsk=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/text v0.37.0 h1:Cqjiwd9eSg8e0QAkyCaQTNHFIIzWtidPahFWR83rTrc=
golang.org/x/text v0.37.0/go.mod h1:a5sjxXGs9hsn/AJVwuElvCAo9v8QYLzvavO5z2PiM38=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
google.golang.org/appengine v1.6.5/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCIDZVag1xfc=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

48
db/hooks/activitypub_actor.go Normal file
View File

@@ -0,0 +1,48 @@
package hooks
import (
"log"
"pocketbase/util"
"time"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/pocketbase/core"
)
func CreateActorHandler(client meilisearch.ServiceManager) func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
err := e.Next()
if err != nil {
return err
}
return util.IndexActors([]*core.Record{e.Record}, client)
}
}
func UpdateActorHandler(client meilisearch.ServiceManager) func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
err := e.Next()
if err != nil {
return err
}
return util.UpdateActor(e.Record, client)
}
}
func DeleteActorHandler(client meilisearch.ServiceManager) func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
task, err := client.Index("actors").DeleteDocument(e.Record.Id, nil)
if err != nil {
return err
}
interval := 500 * time.Millisecond
_, err = client.WaitForTask(task.TaskUID, interval)
if err != nil {
log.Fatalf("Error waiting for task completion: %v", err)
}
return e.Next()
}
}

20
db/hooks/comments.go
View File

@@ -1,6 +1,8 @@
package hooks
import (
"fmt"
"os"
"pocketbase/federation"
"pocketbase/util"
@@ -12,7 +14,23 @@ import (
func CreateCommentHandler() func(e *core.RecordRequestEvent) error {
return func(e *core.RecordRequestEvent) error {
e.Next()
err := e.Next()
if err != nil {
return err
}
// add local iri
origin := os.Getenv("ORIGIN")
if origin == "" {
return fmt.Errorf("ORIGIN not set")
}
if e.Record.GetString("iri") == "" {
e.Record.Set("iri", fmt.Sprintf("%s/api/v1/comment/%s", origin, e.Record.Id))
}
err = e.App.UnsafeWithoutHooks().Save(e.Record)
if err != nil {
return err
}
userActor, err := e.App.FindFirstRecordByData("activitypub_actors", "user", e.Auth.Id)
if err != nil {

146
db/hooks/integrations.go
View File

@@ -1,146 +0,0 @@
package hooks
import (
"encoding/json"
"os"
"pocketbase/util"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
)
func ListIntegrationHandler() func(e *core.RecordsListRequestEvent) error {
return func(e *core.RecordsListRequestEvent) error {
if e.HasSuperuserAuth() {
return e.Next()
}
for _, r := range e.Records {
err := censorIntegrationSecrets(r)
if err != nil {
return err
}
}
return e.Next()
}
}
func CreateIntegrationHandler() func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
err := encryptIntegrationSecrets(e.App, e.Record)
if err != nil {
return err
}
return e.Next()
}
}
func CreateUpdateIntegrationSuccessHandler() func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
err := censorIntegrationSecrets(e.Record)
if err != nil {
return err
}
return e.Next()
}
}
func UpdateIntegrationHandler() func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
err := encryptIntegrationSecrets(e.App, e.Record)
if err != nil {
return err
}
return e.Next()
}
}
func censorIntegrationSecrets(r *core.Record) error {
secrets := map[string][]string{
"strava": {"clientSecret", "refreshToken", "accessToken", "expiresAt"},
"komoot": {"password"},
"hammerhead": {"password"},
}
for key, secretKeys := range secrets {
if integrationString := r.GetString(key); integrationString != "" {
var integration map[string]interface{}
if err := json.Unmarshal([]byte(integrationString), &integration); err != nil {
return err
}
if integration == nil {
continue
}
for _, secretKey := range secretKeys {
integration[secretKey] = ""
}
b, err := json.Marshal(integration)
if err != nil {
return err
}
r.Set(key, string(b))
}
}
return nil
}
func encryptIntegrationSecrets(app core.App, r *core.Record) error {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return apis.NewBadRequestError("POCKETBASE_ENCRYPTION_KEY not set", nil)
}
secrets := map[string][]string{
"strava": {"clientSecret", "refreshToken", "accessToken", "expiresAt"},
"komoot": {"password"},
"hammerhead": {"password"},
}
original, _ := app.FindRecordById("integrations", r.Id)
for key, secretKeys := range secrets {
if integrationString := r.GetString(key); integrationString != "" {
var integration map[string]interface{}
if err := json.Unmarshal([]byte(integrationString), &integration); err != nil {
return err
}
for _, secretKey := range secretKeys {
// If the secret is already encrypted, we don't re-encrypt it.
// TODO: This is a bit of a hack, we should handle this in a more robust way (e.g.
// storing flag on the record or prefixing encrypted strings with enc: or smilar).
// Doing that would also potentially allow us to support key rotation in the future.
if secret, ok := integration[secretKey].(string); ok && len(secret) > 0 && !util.CanDecryptSecret(secret) {
encryptedSecret, err := security.Encrypt([]byte(secret), encryptionKey)
if err != nil {
return err
}
integration[secretKey] = encryptedSecret
} else if original != nil {
originalString := original.GetString(key)
var originalIntegration map[string]interface{}
if err := json.Unmarshal([]byte(originalString), &originalIntegration); err != nil {
return err
}
if integration == nil {
continue
}
integration[secretKey] = originalIntegration[secretKey]
}
}
b, err := json.Marshal(integration)
if err != nil {
return err
}
r.Set(key, string(b))
}
}
return nil
}

30
db/hooks/list.go
View File

@@ -1,6 +1,8 @@
package hooks
import (
"fmt"
"os"
"pocketbase/federation"
"pocketbase/util"
@@ -18,15 +20,20 @@ func CreateListHandler(client meilisearch.ServiceManager) func(e *core.RecordEve
return err
}
if err := util.IndexLists(e.App, []*core.Record{record}, client); err != nil {
// add local iri
origin := os.Getenv("ORIGIN")
if origin == "" {
return fmt.Errorf("ORIGIN not set")
}
if e.Record.GetString("iri") == "" {
e.Record.Set("iri", fmt.Sprintf("%s/api/v1/list/%s", origin, e.Record.Id))
if err = e.App.UnsafeWithoutHooks().Save(e.Record); err != nil {
return err
}
}
if !author.GetBool("isLocal") {
// this happens if someone fetches a remote list
// we create a stub list record for later reference
// no need to create an activity for that
return e.Next()
if err := util.IndexLists(e.App, []*core.Record{record}, client); err != nil {
return err
}
err = e.Next()
@@ -34,6 +41,13 @@ func CreateListHandler(client meilisearch.ServiceManager) func(e *core.RecordEve
return err
}
if !author.GetBool("is_local") {
// this happens if someone fetches a remote list
// we create a stub list record for later reference
// no need to create an activity for that
return nil
}
err = federation.CreateListActivity(e.App, e.Record, pub.CreateType)
if err != nil {
return err
@@ -61,7 +75,7 @@ func UpdateListHandler(client meilisearch.ServiceManager) func(e *core.RecordEve
return err
}
if !author.GetBool("isLocal") {
if !author.GetBool("is_local") {
// this happens if someone fetches a remote list
// we create a stub list record for later reference
// no need to create an activity for that
@@ -73,7 +87,7 @@ func UpdateListHandler(client meilisearch.ServiceManager) func(e *core.RecordEve
return err
}
err = federation.CreateListActivity(e.App, e.Record, pub.CreateType)
err = federation.CreateListActivity(e.App, e.Record, pub.UpdateType)
if err != nil {
return err
}

271
db/hooks/plugin_instances.go Normal file
View File

@@ -0,0 +1,271 @@
package hooks
import (
"encoding/json"
"os"
"github.com/pocketbase/dbx"
"pocketbase/util"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
"pocketbase/pluginsystem"
)
// ListPluginInstanceHandler censors auth values before plugin instances leave
// the API. The database keeps encrypted secrets, but normal users never receive
// the encrypted payload either.
func ListPluginInstanceHandler() func(e *core.RecordsListRequestEvent) error {
return func(e *core.RecordsListRequestEvent) error {
if e.HasSuperuserAuth() {
return e.Next()
}
for _, r := range e.Records {
censorPluginInstanceAuth(e.App, r)
}
return e.Next()
}
}
// ViewPluginInstanceHandler applies the same auth censoring for single-record
// reads that ListPluginInstanceHandler applies for list reads.
func ViewPluginInstanceHandler() func(e *core.RecordRequestEvent) error {
return func(e *core.RecordRequestEvent) error {
if e.HasSuperuserAuth() {
return e.Next()
}
censorPluginInstanceAuth(e.App, e.Record)
return e.Next()
}
}
// CreatePluginInstanceHandler normalizes initial status and encrypts submitted
// auth fields before a plugin instance is persisted.
func CreatePluginInstanceHandler() func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
ensurePluginInstanceStatus(e.Record)
mergePluginInstanceDefaultConfig(e.App, e.Record)
if err := encryptPluginInstanceAuth(e.App, e.Record); err != nil {
return err
}
return e.Next()
}
}
// CreateUpdatePluginInstanceSuccessHandler censors auth values in the response
// body after PocketBase has stored the encrypted values.
func CreateUpdatePluginInstanceSuccessHandler() func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
censorPluginInstanceAuth(e.App, e.Record)
return e.Next()
}
}
// UpdatePluginInstanceHandler re-applies status defaults and encrypts any
// changed auth fields before the update is persisted.
func UpdatePluginInstanceHandler() func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
ensurePluginInstanceStatus(e.Record)
mergePluginInstanceDefaultConfig(e.App, e.Record)
if err := encryptPluginInstanceAuth(e.App, e.Record); err != nil {
return err
}
return e.Next()
}
}
func mergePluginInstanceDefaultConfig(app core.App, r *core.Record) {
defaults := installedPluginDefaultConfig(app, r.GetString("plugin_id"))
if len(defaults) == 0 {
return
}
merged := pluginsystem.CloneJSONMap(defaults)
pluginsystem.MergePluginConfig(merged, pluginsystem.JSONMapFromRecord(r, "config"))
r.Set("config", merged)
}
func installedPluginDefaultConfig(app core.App, pluginID string) map[string]any {
if pluginID == "" {
return map[string]any{}
}
record, _ := app.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": pluginID},
)
if record == nil {
return map[string]any{}
}
return pluginsystem.JSONMapFromRecord(record, "config")
}
func censorPluginInstanceAuth(app core.App, r *core.Record) {
if authString := r.GetString("auth"); authString != "" {
var auth map[string]any
if err := json.Unmarshal([]byte(authString), &auth); err != nil {
r.Set("auth", "{}")
return
}
secretFields := pluginInstanceSecretFields(app, r.GetString("plugin_id"))
encryptAll := len(secretFields) == 0
for key := range auth {
if encryptAll || secretFields[key] {
auth[key] = ""
}
}
b, err := json.Marshal(auth)
if err != nil {
r.Set("auth", "{}")
return
}
r.Set("auth", string(b))
}
}
func ensurePluginInstanceStatus(r *core.Record) {
if r.GetString("status") != "" {
return
}
if r.GetString("auth") == "" {
r.Set("status", "needs_auth")
return
}
if r.GetBool("enabled") {
r.Set("status", "configured")
return
}
r.Set("status", "disabled")
}
func encryptPluginInstanceAuth(app core.App, r *core.Record) error {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return apis.NewBadRequestError("POCKETBASE_ENCRYPTION_KEY not set", nil)
}
authString := r.GetString("auth")
if authString == "" {
return nil
}
var auth map[string]any
if err := json.Unmarshal([]byte(authString), &auth); err != nil {
return err
}
if auth == nil {
return nil
}
var originalAuth map[string]any
if original, _ := app.FindRecordById("plugin_instances", r.Id); original != nil {
originalString := original.GetString("auth")
if originalString != "" {
_ = json.Unmarshal([]byte(originalString), &originalAuth)
}
}
secretFields := pluginInstanceSecretFields(app, r.GetString("plugin_id"))
encryptAll := len(secretFields) == 0
if originalAuth != nil {
for key, value := range originalAuth {
if _, ok := auth[key]; ok {
continue
}
if encryptAll || secretFields[key] {
auth[key] = value
}
}
}
for key, value := range auth {
secret, ok := value.(string)
if !ok {
continue
}
if secret == "" {
if originalAuth != nil {
if restored, ok := originalAuth[key].(string); ok && restored != "" {
secret = restored
}
}
if secret == "" {
continue
}
}
if !encryptAll && !secretFields[key] {
auth[key] = secret
continue
}
if util.CanDecryptSecret(secret) {
auth[key] = secret
continue
}
encryptedSecret, err := security.Encrypt([]byte(secret), encryptionKey)
if err != nil {
return err
}
auth[key] = encryptedSecret
}
b, err := json.Marshal(auth)
if err != nil {
return err
}
r.Set("auth", string(b))
return nil
}
func pluginInstanceSecretFields(app core.App, pluginID string) map[string]bool {
manifest, ok := pluginInstancePluginManifest(app, pluginID)
if !ok {
return nil
}
fields := map[string]bool{}
for _, field := range pluginsystem.InternalAuthSecretFields() {
fields[field] = true
}
for _, context := range manifest.Auth.Contexts {
if context.SecretField != "" {
fields[context.SecretField] = true
}
for _, field := range context.SecretFields {
fields[field] = true
}
}
return fields
}
func pluginInstancePluginManifest(app core.App, pluginID string) (pluginsystem.Manifest, bool) {
record, _ := app.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": pluginID},
)
if record != nil {
var manifest pluginsystem.Manifest
if err := record.UnmarshalJSONField("manifest", &manifest); err == nil && manifest.ID != "" {
return manifest, true
}
}
plugins, err := pluginsystem.LoadLocalPlugins("")
if err != nil {
return pluginsystem.Manifest{}, false
}
for _, plugin := range plugins {
if plugin.Manifest.ID == pluginID {
return plugin.Manifest, true
}
}
return pluginsystem.Manifest{}, false
}

16
db/hooks/summit_logs.go
View File

@@ -1,6 +1,8 @@
package hooks
import (
"fmt"
"os"
"pocketbase/federation"
"pocketbase/util"
@@ -11,12 +13,24 @@ import (
func CreateSummitLogHandler(client meilisearch.ServiceManager) func(e *core.RecordRequestEvent) error {
return func(e *core.RecordRequestEvent) error {
err := e.Next()
if err != nil {
return err
}
// add local iri
origin := os.Getenv("ORIGIN")
if origin == "" {
return fmt.Errorf("ORIGIN not set")
}
if e.Record.GetString("iri") == "" {
e.Record.Set("iri", fmt.Sprintf("%s/api/v1/summit-log/%s", origin, e.Record.Id))
}
err = e.App.UnsafeWithoutHooks().Save(e.Record)
if err != nil {
return err
}
userActor, err := e.App.FindFirstRecordByData("activitypub_actors", "user", e.Auth.Id)
if err != nil {
return err

4
db/hooks/trail_like.go
View File

@@ -51,7 +51,7 @@ func CreateTrailLikeHandler(client meilisearch.ServiceManager) func(e *core.Reco
return err
}
if !actor.GetBool("isLocal") {
if !actor.GetBool("is_local") {
// this happens if someone likes a remote trail
// we create a local copy
// no need to create an activity for that
@@ -107,7 +107,7 @@ func DeleteTrailLikeHandler(client meilisearch.ServiceManager) func(e *core.Reco
return err
}
if !actor.GetBool("isLocal") {
if !actor.GetBool("is_local") {
// this happens if someone likes a remote trail
// we create a local copy
// no need to create an activity for that

40
db/hooks/trails.go
View File

@@ -1,7 +1,9 @@
package hooks
import (
"fmt"
"log"
"os"
"pocketbase/federation"
"pocketbase/util"
"time"
@@ -20,14 +22,24 @@ func CreateTrailHandler(client meilisearch.ServiceManager) func(e *core.RecordEv
if err != nil {
return err
}
if err := util.IndexTrails(e.App, []*core.Record{record}, client); err != nil {
if err := util.SavePolyline(e.App, record); err != nil {
log.Printf("failed to save polyline for trail %s: %v", record.Id, err)
}
// add local iri
origin := os.Getenv("ORIGIN")
if origin == "" {
return fmt.Errorf("ORIGIN not set")
}
if e.Record.GetString("iri") == "" {
e.Record.Set("iri", fmt.Sprintf("%s/api/v1/trail/%s", origin, e.Record.Id))
if err = e.App.UnsafeWithoutHooks().Save(e.Record); err != nil {
return err
}
if !userActor.GetBool("isLocal") {
// this happens if someone fetches a remote trail
// we create a stub trail record for later reference
// no need to create an activity for that
return e.Next()
}
if err := util.IndexTrails(e.App, []*core.Record{record}, client); err != nil {
return err
}
err = e.Next()
@@ -35,6 +47,13 @@ func CreateTrailHandler(client meilisearch.ServiceManager) func(e *core.RecordEv
return err
}
if !userActor.GetBool("is_local") {
// this happens if someone fetches a remote list
// we create a stub list record for later reference
// no need to create an activity for that
return nil
}
ctx, err := util.GetSafeActorContext(nil, userActor)
if err != nil {
@@ -62,11 +81,18 @@ func UpdateTrailHandler(client meilisearch.ServiceManager) func(e *core.RecordEv
if err != nil {
return err
}
if record.GetString("gpx") != record.Original().GetString("gpx") {
if err := util.SavePolyline(e.App, record); err != nil {
log.Printf("failed to save polyline for trail %s: %v", record.Id, err)
}
}
err = util.UpdateTrail(e.App, record, userActor, client)
if err != nil {
return err
}
if !userActor.GetBool("isLocal") {
if !userActor.GetBool("is_local") {
// this happens if someone fetches a remote trail
// we create a stub trail record for later reference
// no need to create an activity for that

22
db/hooks/users.go
View File

@@ -12,36 +12,16 @@ import (
func CreateUserHandler(client meilisearch.ServiceManager) func(e *core.RecordEvent) error {
return func(e *core.RecordEvent) error {
userId := e.Record.Id
err := createDefaultUserSettings(e.App, e.Record.Id)
if err != nil {
return err
}
actor, err := util.ActorFromUser(e.App, e.Record)
_, err = util.ActorFromUser(e.App, e.Record)
if err != nil {
return err
}
searchRules := map[string]interface{}{
"lists": map[string]string{
"filter": "public = true OR author = " + actor.Id + " OR shares = " + userId,
},
"trails": map[string]string{
"filter": "public = true OR author = " + actor.Id + " OR shares = " + userId,
},
}
token, err := util.GenerateMeilisearchToken(searchRules, client)
if err != nil {
return err
}
e.Record.Set("token", token)
if err := e.App.Save(e.Record); err != nil {
return err
}
return e.Next()
}
}

28
db/hooks/waypoint.go Normal file
View File

@@ -0,0 +1,28 @@
package hooks
import (
"fmt"
"os"
"github.com/pocketbase/pocketbase/core"
)
func CreateWaypointHandler() func(e *core.RecordRequestEvent) error {
return func(e *core.RecordRequestEvent) error {
err := e.Next()
if err != nil {
return err
}
// add local iri
origin := os.Getenv("ORIGIN")
if origin == "" {
return fmt.Errorf("ORIGIN not set")
}
if e.Record.GetString("iri") == "" {
e.Record.Set("iri", fmt.Sprintf("%s/api/v1/waypoint/%s", origin, e.Record.Id))
}
return e.App.UnsafeWithoutHooks().Save(e.Record)
}
}

927
db/integrations/hammerhead/hammerhead.go
View File

@@ -1,927 +0,0 @@
package hammerhead
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"mime/multipart"
"net/http"
"math"
"os"
"slices"
"strings"
"time"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/filesystem"
"github.com/pocketbase/pocketbase/tools/security"
"github.com/tkrajina/gpxgo/gpx"
"pocketbase/services/trailmerge"
"pocketbase/util"
)
func SyncHammerhead(app core.App, client meilisearch.ServiceManager) error {
integrations, err := app.FindAllRecords("integrations", dbx.NewExp("true"))
if err != nil {
return err
}
for _, i := range integrations {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return errors.New("POCKETBASE_ENCRYPTION_KEY not set")
}
userId := i.GetString("user")
actor, err := app.FindFirstRecordByData("activitypub_actors", "user", userId)
if err != nil {
warning := fmt.Sprintf("no actor found for user: %s\n", userId)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
ctx, err := util.GetSafeActorContext(nil, actor)
if err != nil {
continue
}
hammerheadString := i.GetString("hammerhead")
hammerheadIntegration := HammerheadIntegration{
Planned: true,
Completed: true,
Merge: trailmerge.DefaultIntegrationAutoMergeSettings(),
}
json.Unmarshal([]byte(hammerheadString), &hammerheadIntegration)
if !hammerheadIntegration.Active || hammerheadIntegration.Email == "" || hammerheadIntegration.Password == "" {
continue
}
h := &HammerheadApi{}
decryptedPassword, err := security.Decrypt(hammerheadIntegration.Password, encryptionKey)
if err != nil {
warning := fmt.Sprintf("unable to decrypt password: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
err = h.Login(hammerheadIntegration.Email, string(decryptedPassword))
if err != nil {
warning := fmt.Sprintf("Hammerhead login failed: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
page := 0
totalPages := 0
stopped := false
var after int64 = 0
if hammerheadIntegration.After != "" {
t, err := time.Parse("2006-01-02", hammerheadIntegration.After)
if err != nil {
return err
}
t = t.UTC()
after = t.Unix()
}
if hammerheadIntegration.Planned {
page = 0
totalPages = 0
stopped = false
for page <= totalPages && !stopped {
curTotalPages := totalPages
tours, curTotalPages, err := h.fetchTours(page)
if err != nil {
warning := fmt.Sprintf("error fetching tours from Hammerhead: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
if curTotalPages > totalPages {
totalPages = curTotalPages
}
err, stopped = syncTrailWithTours(app, client, ctx, h, actor, hammerheadIntegration, tours, after)
if err != nil {
warning := fmt.Sprintf("error syncing Hammerhead tours with trails: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
page += 1
}
}
if hammerheadIntegration.Completed {
page = 0
totalPages = 0
stopped = false
for page <= totalPages && !stopped {
curTotalPages := totalPages
tours, curTotalPages, err := h.fetchActivities(page)
if err != nil {
warning := fmt.Sprintf("error fetching tours from Hammerhead: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
if curTotalPages > totalPages {
totalPages = curTotalPages
}
err, stopped = syncTrailWithActivities(app, client, ctx, h, actor, hammerheadIntegration, tours, after)
if err != nil {
warning := fmt.Sprintf("error syncing Hammerhead tours with trails: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
page += 1
}
}
}
return nil
}
type BasicAuthToken struct {
Key string
Value string
}
func (b BasicAuthToken) Apply(req *http.Request) {
req.Header.Set("Authorization", "Bearer "+b.Value)
}
type HammerheadApi struct {
UserID string
Token string
}
func (h *HammerheadApi) buildHeader() *BasicAuthToken {
if h.UserID != "" && h.Token != "" {
return &BasicAuthToken{h.UserID, h.Token}
}
return nil
}
func getToken(uri string, auth *BasicAuthToken) ([]byte, error) {
client := &http.Client{}
var jsonStr = []byte(`{"grant_type": "password", "username": "` + auth.Key + `", "password": "` + auth.Value + `"}`)
req, err := http.NewRequest("POST", uri, bytes.NewBuffer(jsonStr))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("error retrieving auth token from Hammerhead (%d): %s", resp.StatusCode, string(body))
}
return io.ReadAll(resp.Body)
}
func (h *HammerheadApi) UploadActivities(e *core.RequestEvent) error {
files, err := e.FindUploadedFiles("file")
if err != nil {
if errors.Is(err, http.ErrMissingFile) {
return apis.NewBadRequestError("file field is required", err)
}
return apis.NewBadRequestError("invalid multipart payload", err)
}
if len(files) == 0 {
return apis.NewBadRequestError("file field is required", nil)
}
fileToUpload := files[0]
reader, err := fileToUpload.Reader.Open()
if err != nil {
return err
}
defer reader.Close()
var buf bytes.Buffer
writer := multipart.NewWriter(&buf)
part, err := writer.CreateFormFile("file", fileToUpload.OriginalName)
if err != nil {
return err
}
if _, err := io.Copy(part, reader); err != nil {
return err
}
contentType := writer.FormDataContentType()
if err := writer.Close(); err != nil {
return err
}
currentURI := fmt.Sprintf("https://dashboard.hammerhead.io/v1/users/%s/routes/import/file", h.UserID)
if _, err := sendPostRequest(currentURI, &buf, contentType, h.buildHeader()); err != nil {
return err
}
return nil
}
func sendPostRequest(url string, body io.Reader, contentType string, auth *BasicAuthToken) ([]byte, error) {
client := &http.Client{}
req, err := http.NewRequest("POST", url, body)
if err != nil {
return nil, err
}
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
if auth != nil {
auth.Apply(req)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("error sending request to Hammerhead (%d): %s", resp.StatusCode, string(body))
}
return io.ReadAll(resp.Body)
}
func sendGetRequest(url string, auth *BasicAuthToken) ([]byte, error) {
client := &http.Client{}
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
if auth != nil {
auth.Apply(req)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("error sending request to Hammerhead (%d): %s", resp.StatusCode, string(body))
}
return io.ReadAll(resp.Body)
}
func (h *HammerheadApi) Login(email, password string) error {
url := "https://dashboard.hammerhead.io/v1/auth/token"
body, err := getToken(url, &BasicAuthToken{email, password})
if err != nil {
return err
}
var data LoginResponse
json.Unmarshal(body, &data)
h.Token = data.Token
derivedUserID, err := extractUserIDFromToken(data.Token)
if err != nil {
return fmt.Errorf("unable to determine Hammerhead user id automatically: %w", err)
}
h.UserID = derivedUserID
return nil
}
func extractUserIDFromToken(token string) (string, error) {
parts := strings.Split(token, ".")
if len(parts) < 2 {
return "", errors.New("token is not a JWT")
}
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return "", fmt.Errorf("unable to decode JWT payload: %w", err)
}
var claims map[string]any
if err := json.Unmarshal(payload, &claims); err != nil {
return "", fmt.Errorf("unable to decode JWT claims: %w", err)
}
if value, ok := claims["sub"].(string); ok && value != "" {
return value, nil
}
return "", errors.New("no sub claim found in token")
}
func (h *HammerheadApi) fetchActivities(page int) ([]HammerheadActivityResponse, int, error) {
currentUri := fmt.Sprintf("https://dashboard.hammerhead.io/v1/users/%s/activities?perPage=50&page=%d&search=&orderBy=NEWEST&ascending=true", h.UserID, page)
body, err := sendGetRequest(currentUri, h.buildHeader())
if err != nil {
return nil, 0, err
}
var data HammerheadActivitiesResponse
json.Unmarshal(body, &data)
tours := data.Tours
return tours, data.Pages, nil
}
func (h *HammerheadApi) fetchTours(page int) ([]HammerheadTourResponse, int, error) {
currentUri := fmt.Sprintf("https://dashboard.hammerhead.io/v1/users/%s/routes?perPage=50&page=%d&search=&orderBy=NEWEST&ascending=true&exclude=archive", h.UserID, page)
body, err := sendGetRequest(currentUri, h.buildHeader())
if err != nil {
return nil, 0, err
}
var data HammerheadToursResponse
json.Unmarshal(body, &data)
tours := data.Data
return tours, data.TotalPages, nil
}
func (h *HammerheadApi) fetchDetailedActivity(tour HammerheadActivityResponse) (*HammerheadActivity, error) {
url := fmt.Sprintf("https://dashboard.hammerhead.io/v1/users/%s/activities/%s/details", h.UserID, tour.ID)
body, err := sendGetRequest(url, h.buildHeader())
if err != nil {
return nil, err
}
var data *HammerheadActivity
json.Unmarshal(body, &data)
return data, nil
}
func (h *HammerheadApi) fetchDetailedTour(tour HammerheadTourResponse) (*HammerheadTour, error) {
url := fmt.Sprintf("https://dashboard.hammerhead.io/v1/users/%s/routes/%s", h.UserID, tour.ID)
body, err := sendGetRequest(url, h.buildHeader())
if err != nil {
return nil, err
}
var data *HammerheadTour
json.Unmarshal(body, &data)
return data, nil
}
func syncTrailWithTours(app core.App, client meilisearch.ServiceManager, ctx context.Context, k *HammerheadApi, actor *core.Record, integration HammerheadIntegration, tours []HammerheadTourResponse, after int64) (error, bool) {
for _, tour := range tours {
existingTrail, err := util.FindTrailByExternalReference(app, "hammerhead", tour.ID)
if err != nil {
return err, true
}
if existingTrail != nil {
continue
}
detailedTour, err := k.fetchDetailedTour(tour)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to fetch details for tour '%s': %v", tour.Name, err))
continue
}
if detailedTour.CreatedAt.Unix() < after {
return nil, true
}
if detailedTour.Distance <= 0 {
app.Logger().Warn(fmt.Sprintf("Skipping Hammerhead tour '%s' with zero distance", tour.Name))
continue
}
gpx, err := generateTourGPX(detailedTour)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to generate GPX for tour '%s': %v", tour.Name, err))
continue
}
trailID, err := createTrailFromTour(app, detailedTour, gpx, actor.Id)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create trail for tour '%s': %v", tour.Name, err))
continue
}
if err := trailmerge.TryAutoMergeImportedTrail(app, client, ctx, actor, trailID, integration.Merge); err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to auto-merge imported Hammerhead tour '%s': %v", tour.Name, err))
}
}
return nil, false
}
func syncTrailWithActivities(app core.App, client meilisearch.ServiceManager, ctx context.Context, k *HammerheadApi, actor *core.Record, integration HammerheadIntegration, tours []HammerheadActivityResponse, after int64) (error, bool) {
for _, tour := range tours {
existingTrail, err := util.FindTrailByExternalReference(app, "hammerhead", tour.ID)
if err != nil {
return err, true
}
if existingTrail != nil {
continue
}
detailedTour, err := k.fetchDetailedActivity(tour)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to fetch details for tour '%s': %v", tour.Name, err))
continue
}
if detailedTour.ActivityData.CreatedAt.Unix() < after {
return nil, true
}
distance, ok := activityDistance(detailedTour)
if !ok || distance <= 0 {
app.Logger().Warn(fmt.Sprintf("Skipping Hammerhead activity '%s' with zero distance", tour.Name))
continue
}
gpx, err := generateActivityGPX(detailedTour)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to generate GPX for tour '%s': %v", tour.Name, err))
continue
}
trailID, err := createTrailFromActivity(app, detailedTour, gpx, actor.Id)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create trail for tour '%s': %v", tour.Name, err))
continue
}
if err := trailmerge.TryAutoMergeImportedTrail(app, client, ctx, actor, trailID, integration.Merge); err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to auto-merge imported Hammerhead activity '%s': %v", tour.Name, err))
}
}
return nil, false
}
func activityDistance(detailedTour *HammerheadActivity) (float64, bool) {
idDistance := slices.IndexFunc(detailedTour.ActivityData.ActivityInfo, func(c HammerheadInfo) bool { return c.Key == "TYPE_DISTANCE_ID" })
if idDistance < 0 {
return 0, false
}
return detailedTour.ActivityData.ActivityInfo[idDistance].Value.Value, true
}
func createTrailFromActivity(app core.App, detailedTour *HammerheadActivity, gpx *filesystem.File, actor string) (string, error) {
trailid := security.RandomStringWithAlphabet(core.DefaultIdLength, core.DefaultIdAlphabet)
collection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return "", err
}
record := core.NewRecord(collection)
category, _ := app.FindFirstRecordByData("categories", "name", "Biking" /*ToDo: Mapping*/)
categoryId := ""
if category != nil {
categoryId = category.Id
}
diffculty := "easy" // ToDo: calculate difficulty
idDistance := slices.IndexFunc(detailedTour.ActivityData.ActivityInfo, func(c HammerheadInfo) bool { return c.Key == "TYPE_DISTANCE_ID" })
idElevationGain := slices.IndexFunc(detailedTour.ActivityData.ActivityInfo, func(c HammerheadInfo) bool { return c.Key == "TYPE_ELEVATION_GAIN_ID" })
idElevationLoss := slices.IndexFunc(detailedTour.ActivityData.ActivityInfo, func(c HammerheadInfo) bool { return c.Key == "TYPE_ELEVATION_LOSS_ID" })
duration := 0
for _, lap := range detailedTour.ActivityData.Laps {
duration += lap.ActiveTime
}
startLat := float64(0)
startLng := float64(0)
for i, lat := range detailedTour.RecordData.Lat {
if lat != float64(0) {
startLat = lat
startLng = detailedTour.RecordData.Lng[i]
break
}
}
record.Load(map[string]any{
"id": trailid,
"name": detailedTour.ActivityData.Name,
"public": false,
"completed": true,
"distance": detailedTour.ActivityData.ActivityInfo[idDistance].Value.Value,
"elevation_gain": detailedTour.ActivityData.ActivityInfo[idElevationGain].Value.Value,
"elevation_loss": detailedTour.ActivityData.ActivityInfo[idElevationLoss].Value.Value,
"duration": duration / 1000,
"date": detailedTour.ActivityData.CreatedAt,
"external_provider": "hammerhead",
"external_id": detailedTour.ActivityData.ID,
"lat": startLat,
"lon": startLng,
"difficulty": diffculty,
"category": categoryId,
"author": actor,
})
if gpx != nil {
record.Set("gpx", gpx)
}
if err := app.Save(record); err != nil {
return "", err
}
if err := util.EnsureTrailExternalReference(app, trailid, "hammerhead", detailedTour.ActivityData.ID); err != nil {
return "", err
}
collection, err = app.FindCollectionByNameOrId("summit_logs")
if err != nil {
return "", err
}
summitLogRecord := core.NewRecord(collection)
summitLogRecord.Load(map[string]any{
"distance": detailedTour.ActivityData.ActivityInfo[idDistance].Value.Value,
"elevation_gain": detailedTour.ActivityData.ActivityInfo[idElevationGain].Value.Value,
"elevation_loss": detailedTour.ActivityData.ActivityInfo[idElevationLoss].Value.Value,
"duration": duration / 1000,
"date": detailedTour.ActivityData.CreatedAt,
"author": actor,
"trail": trailid,
})
if err := app.Save(summitLogRecord); err != nil {
return "", err
}
return trailid, nil
}
func createTrailFromTour(app core.App, detailedTour *HammerheadTour, gpx *filesystem.File, actor string) (string, error) {
trailid := security.RandomStringWithAlphabet(core.DefaultIdLength, core.DefaultIdAlphabet)
collection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return "", err
}
record := core.NewRecord(collection)
category, _ := app.FindFirstRecordByData("categories", "name", "Biking" /*ToDo: Mapping*/)
categoryId := ""
if category != nil {
categoryId = category.Id
}
diffculty := "easy" // ToDo: calculate difficulty
record.Load(map[string]any{
"id": trailid,
"name": detailedTour.Name,
"public": detailedTour.IsPublic,
"distance": detailedTour.Distance,
"elevation_gain": detailedTour.Elevation.Gain,
"elevation_loss": detailedTour.Elevation.Loss,
"date": detailedTour.CreatedAt,
"lat": detailedTour.StartLocation.Lat,
"lon": detailedTour.StartLocation.Lng,
"difficulty": diffculty,
"category": categoryId,
"author": actor,
})
if gpx != nil {
record.Set("gpx", gpx)
}
if err := app.Save(record); err != nil {
return "", err
}
if err := util.EnsureTrailExternalReference(app, trailid, "hammerhead", detailedTour.ID); err != nil {
return "", err
}
return trailid, nil
}
func generateActivityGPX(detailedTour *HammerheadActivity) (*filesystem.File, error) {
times := len(detailedTour.RecordData.Timestamp)
if times == 0 {
return nil, nil
}
var points []gpx.GPXPoint
const zeroEps = 1e-4
// iterate over timestamps and only add points when lat/lng exist for the same index
for i := 0; i < times; i++ {
// ensure we have latitude and longitude for this index
if i < len(detailedTour.RecordData.Lat) && i < len(detailedTour.RecordData.Lng) {
lat := detailedTour.RecordData.Lat[i]
lng := detailedTour.RecordData.Lng[i]
// exclude near (0,0) garbage points
if math.Abs(lat) < zeroEps && math.Abs(lng) < zeroEps {
continue
}
t := detailedTour.RecordData.Timestamp[i]
elevation := float64(0)
if i < len(detailedTour.RecordData.Elevation) {
elevation = detailedTour.RecordData.Elevation[i] / 1000.0
}
points = append(points, gpx.GPXPoint{
Point: gpx.Point{
Latitude: lat,
Longitude: lng,
Elevation: *gpx.NewNullableFloat64(elevation),
},
Timestamp: time.Unix(int64(t), 0),
})
}
}
if len(points) == 0 {
return nil, nil
}
gpxData := &gpx.GPX{
Version: "1.1",
Creator: "Hammerhead GPX Exporter",
Tracks: []gpx.GPXTrack{
{
Name: detailedTour.ActivityData.Name,
Segments: []gpx.GPXTrackSegment{
{
Points: points,
},
},
},
},
}
gpxAsXML, err := gpxData.ToXml(gpx.ToXmlParams{Version: "1.1", Indent: true})
if err != nil {
return nil, err
}
gpxFile, err := filesystem.NewFileFromBytes(gpxAsXML, detailedTour.ActivityData.Name+".gpx")
if err != nil {
return nil, err
}
return gpxFile, nil
}
func generateTourGPX(detailedTour *HammerheadTour) (*filesystem.File, error) {
poly := detailedTour.RoutePolyline
coords, err := decodePolyline(poly)
if err != nil {
return nil, fmt.Errorf("decode polyline: %w", err)
}
if len(coords) == 0 {
return nil, nil
}
// try to get elevation polyline (adjust field path if your struct differs)
elevations := []float64{}
// precision 100 is common for Valhalla elevation encodings; change if needed
if decoded, err := decodeElevations(detailedTour.Elevation.Polyline, 100000); err == nil {
elevations = decoded
}
// Heuristic: detect if coords are (lng,lat) instead of (lat,lng).
// Count how many points look valid in each orientation and pick the best.
validAsLat := 0
validAsLng := 0
for _, c := range coords {
// treat c[0] as lat, c[1] as lng
if c[0] >= -90 && c[0] <= 90 && c[1] >= -180 && c[1] <= 180 {
validAsLat++
}
// treat c[1] as lat, c[0] as lng (swapped)
if c[1] >= -90 && c[1] <= 90 && c[0] >= -180 && c[0] <= 180 {
validAsLng++
}
}
swap := false
if validAsLng > validAsLat {
swap = true
}
var points []gpx.GPXPoint
for i, c := range coords {
lat := c[0]
lng := c[1]
if swap {
lat, lng = c[1], c[0]
}
// choose elevation:
elevation := 0.0
if len(elevations) == len(coords) {
elevation = elevations[i]
} else if len(elevations) > 0 {
// map index proportionally if lengths differ
j := int(math.Round(float64(i) * float64(len(elevations)-1) / float64(len(coords)-1)))
if j < 0 {
j = 0
}
if j >= len(elevations) {
j = len(elevations) - 1
}
elevation = elevations[j]
}
points = append(points, gpx.GPXPoint{
Point: gpx.Point{
Latitude: lat,
Longitude: lng,
Elevation: *gpx.NewNullableFloat64(elevation),
},
})
}
gpxData := &gpx.GPX{
Version: "1.1",
Creator: "Hammerhead GPX Exporter",
Tracks: []gpx.GPXTrack{
{
Name: detailedTour.Name,
Segments: []gpx.GPXTrackSegment{
{
Points: points,
},
},
},
},
}
gpxAsXML, err := gpxData.ToXml(gpx.ToXmlParams{Version: "1.1", Indent: true})
if err != nil {
return nil, err
}
gpxFile, err := filesystem.NewFileFromBytes(gpxAsXML, detailedTour.Name+".gpx")
if err != nil {
return nil, err
}
return gpxFile, nil
}
// decodePolyline decodes an encoded polyline string (Google Polyline Algorithm)
// returns slice of [lat, lng] pairs (precision 1e5).
func decodePolyline(s string) ([][2]float64, error) {
if s == "" {
return nil, nil
}
var coords [][2]float64
index := 0
lat := 0
lng := 0
for index < len(s) {
// decode latitude
result := 0
shift := uint(0)
for {
if index >= len(s) {
return nil, fmt.Errorf("invalid polyline encoding")
}
b := int(s[index]) - 63
index++
result |= (b & 0x1F) << shift
shift += 5
if b < 0x20 {
break
}
}
dlat := (result >> 1) ^ (-(result & 1))
lat += dlat
// decode longitude
result = 0
shift = 0
for {
if index >= len(s) {
return nil, fmt.Errorf("invalid polyline encoding")
}
b := int(s[index]) - 63
index++
result |= (b & 0x1F) << shift
shift += 5
if b < 0x20 {
break
}
}
dlng := (result >> 1) ^ (-(result & 1))
lng += dlng
coords = append(coords, [2]float64{float64(lat) / 1e5, float64(lng) / 1e5})
}
// Auto-normalize scale if values are out of realistic lat/lon ranges.
// Some providers use different precision/scales; repeatedly divide by 10
// until all values fit into valid ranges.
if len(coords) > 0 {
maxLat := 0.0
maxLng := 0.0
for _, c := range coords {
if abs := math.Abs(c[0]); abs > maxLat {
maxLat = abs
}
if abs := math.Abs(c[1]); abs > maxLng {
maxLng = abs
}
}
// If values are too large (e.g. > 90 lat or > 180 lon), rescale down.
for (maxLat > 90.0 || maxLng > 180.0) && (maxLat > 0 && maxLng > 0) {
for i := range coords {
coords[i][0] /= 10.0
coords[i][1] /= 10.0
}
maxLat /= 10.0
maxLng /= 10.0
}
}
return coords, nil
}
// decodeElevations decodes a single-dimension delta-encoded polyline string.
// precision is the divisor (e.g. 100 for centi-meters -> meters). Returns elevation values in same units as precision (meters if precision=100).
func decodeElevations(s string, precision float64) ([]float64, error) {
if s == "" {
return nil, nil
}
var elevs []float64
index := 0
val := 0
for index < len(s) {
result := 0
shift := uint(0)
for {
if index >= len(s) {
return nil, fmt.Errorf("invalid elevation encoding")
}
b := int(s[index]) - 63
index++
result |= (b & 0x1F) << shift
shift += 5
if b < 0x20 {
break
}
}
d := (result >> 1) ^ (-(result & 1))
val += d
elevs = append(elevs, float64(val)/precision)
}
return elevs, nil
}

209
db/integrations/hammerhead/models.go
View File

@@ -1,209 +0,0 @@
package hammerhead
import (
"time"
"pocketbase/services/trailmerge"
)
type HammerheadToursResponse struct {
TotalItems int `json:"totalItems"`
TotalPages int `json:"totalPages"`
PerPage int `json:"perPage"`
CurrentPage int `json:"currentPage"`
Data []HammerheadTourResponse `json:"data"`
}
type HammerheadTourResponse struct {
StartLocationName string `json:"startLocationName"`
IsAutoImported bool `json:"isAutoImported"`
SummaryPolyline string `json:"summaryPolyline"`
IsStarred bool `json:"isStarred"`
IsPublic bool `json:"isPublic"`
Collections any `json:"collections"`
Gain int `json:"gain"`
Distance float64 `json:"distance"`
Name string `json:"name"`
RoutingType string `json:"routingType"`
ID string `json:"id"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
Source string `json:"source"`
}
type HammerheadTourElevation struct {
Gain float64 `json:"gain"`
Loss float64 `json:"loss"`
Min float64 `json:"min"`
Max float64 `json:"max"`
Source string `json:"source"`
Polyline string `json:"polyline"`
}
type HammerheadLocation struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
}
type HammerheadWaypoint struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
WaypointType string `json:"waypointType"`
PolylineIndex int `json:"polylineIndex"`
}
type HammerheadTour struct {
ID string `json:"id"`
CreatedAt time.Time `json:"createdAt"`
Name string `json:"name"`
Distance float64 `json:"distance"`
Elevation HammerheadTourElevation `json:"elevation"`
IsStarred bool `json:"isStarred"`
StartLocationName string `json:"startLocationName"`
EndLocationName string `json:"endLocationName"`
StartLocation HammerheadLocation `json:"startLocation"`
EndLocation HammerheadLocation `json:"endLocation"`
Waypoints []HammerheadWaypoint `json:"waypoints"`
Collections []string `json:"collections"`
RoutePolyline string `json:"routePolyline"`
SummaryPolyline string `json:"summaryPolyline"`
Source string `json:"source"`
SourceID string `json:"sourceId"`
IsPublic bool `json:"isPublic"`
ImageVersion string `json:"imageVersion"`
IsAutoImported bool `json:"isAutoImported"`
UpdatedAt time.Time `json:"updatedAt"`
Bounds []HammerheadLocation `json:"bounds"`
}
type HammerheadIntegration struct {
Active bool `json:"active"`
Email string `json:"email"`
Password string `json:"password"`
Planned bool `json:"planned"`
Completed bool `json:"completed"`
After string `json:"after,omitempty"`
Merge trailmerge.IntegrationAutoMergeSettings `json:"merge"`
}
type LoginResponse struct {
Token string `json:"access_token"`
Type string `json:"token_type"`
Expires int `json:"expires_in"`
}
type HammerheadActivitiesResponse struct {
Items int `json:"totalItems"`
Pages int `json:"totalPages"`
PerPage int `json:"perPage"`
Tours []HammerheadActivityResponse `json:"data"`
}
type HammerheadActivityResponse struct {
ID string `json:"id"`
CreatedAt time.Time `json:"createdAt"`
Name string `json:"name"`
Client string `json:"client"`
ActiveTime int `json:"activeTime"`
Duration HammerheadTourDuration `json:"duration"`
Sync HammerheadSync `json:"partners"`
ActivityInfo []HammerheadInfo `json:"activityInfo"`
}
type HammerheadInfoValue struct {
Format string `json:"format"`
Value float64 `json:"value"`
}
type HammerheadInfo struct {
Key string `json:"key"`
Value HammerheadInfoValue `json:"value"`
}
type HammerheadPartner struct {
Partner string `json:"partner"`
NeedsUpload bool `json:"needsUpload"`
ExternalID string `json:"externalId"`
Attempts int `json:"attempts"`
UploadedAt time.Time `json:"uploadedAt"`
}
type HammerheadSync struct {
Description string `json:"description"`
Tags []any `json:"tags"`
Synced bool `json:"synced"`
Partners []HammerheadPartner `json:"partners"`
}
type HammerheadTourDuration struct {
ElapsedTime int `json:"elapsedTime"`
StartTime time.Time `json:"startTime"`
EndTime time.Time `json:"endTime"`
}
type HammerheadActivity struct {
ActivityData HammerheadActivityData `json:"activityData"`
SessionData HammerheadSessionData `json:"sessionData"`
RecordData HammerheadRecordData `json:"recordData"`
ShiftData HammerheadShiftData `json:"shiftData"`
LapData HammerheadLapData `json:"lapData"`
DeviceBatteryData HammerheadDeviceBatteryData `json:"deviceBatteryData"`
}
type HammerheadDuration struct {
ElapsedTime int `json:"elapsedTime"`
StartTime time.Time `json:"startTime"`
EndTime time.Time `json:"endTime"`
}
type HammerheadLapDetail struct {
ActiveTime int `json:"activeTime"`
Duration HammerheadDuration `json:"duration"`
LapNumber int `json:"lapNumber"`
Pauses []HammerheadDuration `json:"pauses"`
LapInfo []HammerheadInfo `json:"lapInfo"`
Trigger string `json:"trigger"`
}
type HammerheadActivityData struct {
ID string `json:"id"`
Name string `json:"name"`
BikeID string `json:"bikeId"`
Client string `json:"client"`
ActiveTime int `json:"activeTime"`
Duration HammerheadDuration `json:"duration"`
ActivityInfo []HammerheadInfo `json:"activityInfo"`
Laps []HammerheadLapDetail `json:"laps"`
Polyline string `json:"polyline"`
Sync HammerheadSync `json:"sync"`
ActivityType string `json:"activityType"`
Climbs []HammerheadClimb `json:"climbs"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
type HammerheadClimb struct {
StartDistance float64 `json:"startDistance"`
EndDistance float64 `json:"endDistance"`
Distance float64 `json:"distance"`
}
type HammerheadSessionData struct {
ThresholdPower int `json:"thresholdPower"`
FrontGears []int `json:"frontGears"`
RearGears []int `json:"rearGears"`
}
type HammerheadRecordData struct {
Distance []float64 `json:"distance"`
Timestamp []int `json:"timestamp"`
Elevation []float64 `json:"elevation"`
Grade []float64 `json:"grade"`
Lat []float64 `json:"lat"`
Lng []float64 `json:"lng"`
Speed []float64 `json:"speed"`
Power []any `json:"power"`
Temperature []int `json:"temperature"`
}
type HammerheadShiftData struct {
Timestamp []int `json:"timestamp"`
FrontChange []bool `json:"frontChange"`
FrontGear []int `json:"frontGear"`
RearGear []int `json:"rearGear"`
FrontGearNum []int `json:"frontGearNum"`
RearGearNum []int `json:"rearGearNum"`
}
type HammerheadLapData struct {
Timestamp []int `json:"timestamp"`
Trigger []string `json:"trigger"`
}
type HammerheadDeviceBatteryData struct {
Timestamp []int `json:"timestamp"`
DeviceBattery []int `json:"deviceBattery"`
}

510
db/integrations/komoot/komoot.go
View File

@@ -1,510 +0,0 @@
package komoot
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"strings"
"time"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/filesystem"
"github.com/pocketbase/pocketbase/tools/security"
"github.com/tkrajina/gpxgo/gpx"
"pocketbase/services/trailmerge"
"pocketbase/util"
)
func SyncKomoot(app core.App, client meilisearch.ServiceManager) error {
integrations, err := app.FindAllRecords("integrations", dbx.NewExp("true"))
if err != nil {
return err
}
for _, i := range integrations {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return errors.New("POCKETBASE_ENCRYPTION_KEY not set")
}
userId := i.GetString("user")
actor, err := app.FindFirstRecordByData("activitypub_actors", "user", userId)
if err != nil {
warning := fmt.Sprintf("no actor found for user: %s\n", userId)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
ctx, err := util.GetSafeActorContext(nil, actor)
if err != nil {
continue
}
komootString := i.GetString("komoot")
komootIntegration := KomootIntegration{
Planned: true,
Completed: true,
Merge: trailmerge.DefaultIntegrationAutoMergeSettings(),
}
json.Unmarshal([]byte(komootString), &komootIntegration)
if !komootIntegration.Active || komootIntegration.Email == "" || komootIntegration.Password == "" {
continue
}
k := &KomootApi{}
decryptedPassword, err := security.Decrypt(komootIntegration.Password, encryptionKey)
if err != nil {
warning := fmt.Sprintf("unable to decrypt password: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
err = k.Login(komootIntegration.Email, string(decryptedPassword))
if err != nil {
warning := fmt.Sprintf("komoot login failed: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
totalPages := 1
for page := 0; page < totalPages; page++ {
tours, tp, err := k.fetchTours(page)
if err != nil {
warning := fmt.Sprintf("error fetching tours from komoot (page %d): %v\n", page, err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
totalPages = tp
allAlreadySynced, err := syncTrailWithTours(app, client, ctx, k, komootIntegration, userId, actor, tours)
if err != nil {
warning := fmt.Sprintf("error syncing komoot tours with trails: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
if allAlreadySynced {
break
}
}
}
return nil
}
type BasicAuthToken struct {
Key string
Value string
}
func (b BasicAuthToken) Apply(req *http.Request) {
authStr := "Basic " + base64.StdEncoding.EncodeToString([]byte(b.Key+":"+b.Value))
req.Header.Set("Authorization", authStr)
}
type KomootApi struct {
UserID string
Token string
}
func (k *KomootApi) buildHeader() *BasicAuthToken {
if k.UserID != "" && k.Token != "" {
return &BasicAuthToken{k.UserID, k.Token}
}
return nil
}
func sendRequest(url string, auth *BasicAuthToken) ([]byte, error) {
client := &http.Client{}
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
if auth != nil {
auth.Apply(req)
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("error sending request to komoot (%d): %s", resp.StatusCode, string(body))
}
return io.ReadAll(resp.Body)
}
func (k *KomootApi) Login(email, password string) error {
url := fmt.Sprintf("https://api.komoot.de/v006/account/email/%s/", email)
body, err := sendRequest(url, &BasicAuthToken{email, password})
if err != nil {
return err
}
var data LoginResponse
json.Unmarshal(body, &data)
k.UserID = data.Username
k.Token = data.Password
return nil
}
func (k *KomootApi) fetchTours(page int) ([]KomootTour, int, error) {
currentUri := fmt.Sprintf("https://api.komoot.de/v007/users/%s/tours/?page=%d&sort_field=date&sort_direction=desc&limit=30", k.UserID, page)
body, err := sendRequest(currentUri, k.buildHeader())
if err != nil {
return nil, 0, err
}
var data KomootToursResponse
json.Unmarshal(body, &data)
return data.Embedded.Tours, data.Page.TotalPages, nil
}
func (k *KomootApi) fetchDetailedTour(tour KomootTour) (*DetailedKomootTour, error) {
url := fmt.Sprintf("https://api.komoot.de/v007/tours/%d?_embedded=coordinates,way_types,surfaces,directions,participants,timeline,cover_images&directions=v2&fields=timeline&format=coordinate_array&timeline_highlights_fields=tips,recommenders&page=2", tour.ID)
body, err := sendRequest(url, k.buildHeader())
if err != nil {
return nil, err
}
var data *DetailedKomootTour
json.Unmarshal(body, &data)
return data, nil
}
// syncTrailWithTours imports tours not yet in the DB. Returns allAlreadySynced=true
// when every tour on this page was already imported, so the caller can stop paginating
// early during incremental syncs. Tours skipped due to type filters do NOT count as
// synced - only tours already present in the DB do.
func syncTrailWithTours(app core.App, client meilisearch.ServiceManager, ctx context.Context, k *KomootApi, i KomootIntegration, user string, actor *core.Record, tours []KomootTour) (bool, error) {
allAlreadySynced := true
for _, tour := range tours {
existingTrail, err := util.FindTrailByExternalReference(app, "komoot", strconv.Itoa(int(tour.ID)))
if err != nil {
return false, err
}
if existingTrail != nil {
continue
}
// Tour is not yet in the DB - we must keep paginating regardless of type filter
allAlreadySynced = false
if (tour.Type == "tour_planned" && !i.Planned) || (tour.Type == "tour_recorded" && !i.Completed) {
continue
}
detailedTour, err := k.fetchDetailedTour(tour)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to fetch details for tour '%s': %v", tour.Name, err))
continue
}
gpx, err := generateTourGPX(detailedTour)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to generate GPX for tour '%s': %v", tour.Name, err))
continue
}
trailid, err := createTrailFromTour(app, k, detailedTour, gpx, user, actor.Id, i.Privacy)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create trail for tour '%s': %v", tour.Name, err))
continue
}
err = createWaypointsFromTour(app, detailedTour, user, trailid)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create waypoints for tour '%s': %v", tour.Name, err))
continue
}
if err := trailmerge.TryAutoMergeImportedTrail(app, client, ctx, actor, trailid, i.Merge); err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to auto-merge imported komoot tour '%s': %v", tour.Name, err))
}
}
return allAlreadySynced, nil
}
func createTrailFromTour(app core.App, k *KomootApi, detailedTour *DetailedKomootTour, gpx *filesystem.File, user string, actor string, privacy string) (string, error) {
trailid := security.RandomStringWithAlphabet(core.DefaultIdLength, core.DefaultIdAlphabet)
collection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return "", err
}
record := core.NewRecord(collection)
categoryMap := map[string]string{
"hike": "Hiking",
"touringbicycle": "Biking",
"mtb": "Biking",
"racebike": "Biking",
"jogging": "Walking",
"mtb_easy": "Workout",
"mtb_advanced": "Walking",
"mountaineering": "Hiking",
}
category, _ := app.FindFirstRecordByData("categories", "name", categoryMap[detailedTour.Sport])
categoryId := ""
if category != nil {
categoryId = category.Id
}
var photos []*filesystem.File
if len(detailedTour.Embedded.CoverImages.Embedded.Items) > 0 {
photos, err = fetchRoutePhotos(k, detailedTour)
if err != nil {
return "", err
}
} else {
photo, err := fetchPhoto(detailedTour.MapImage.Src, "", "")
if err != nil {
return "", err
}
photos = append(photos, photo)
}
diffculty := detailedTour.Difficulty.Grade
if diffculty == "" {
diffculty = "easy"
}
public := detailedTour.Status == "public"
if privacy == "settings" {
privacySettings := struct {
Trails string `json:"trails"`
}{}
settings, _ := app.FindFirstRecordByData("settings", "user", user)
err = settings.UnmarshalJSONField("privacy", &privacySettings)
if err != nil {
return "", err
}
public = privacySettings.Trails == "public"
}
record.Load(map[string]any{
"id": trailid,
"name": detailedTour.Name,
"public": public,
"completed": detailedTour.Type == "tour_recorded",
"distance": detailedTour.Distance,
"elevation_gain": detailedTour.ElevationUp,
"elevation_loss": detailedTour.ElevationDown,
"duration": detailedTour.Duration,
"date": detailedTour.Date,
"external_provider": "komoot",
"external_id": strconv.Itoa(detailedTour.ID),
"lat": detailedTour.StartPoint.Lat,
"lon": detailedTour.StartPoint.Lng,
"difficulty": diffculty,
"category": categoryId,
"author": actor,
})
if photos != nil {
record.Set("photos", photos)
}
if gpx != nil {
record.Set("gpx", gpx)
}
if err := app.Save(record); err != nil {
return "", err
}
if err := util.EnsureTrailExternalReference(app, trailid, "komoot", strconv.Itoa(detailedTour.ID)); err != nil {
return "", err
}
if detailedTour.Type == "tour_recorded" {
collection, err := app.FindCollectionByNameOrId("summit_logs")
if err != nil {
return "", err
}
summitLogRecord := core.NewRecord(collection)
summitLogRecord.Load(map[string]any{
"distance": detailedTour.Distance,
"elevation_gain": detailedTour.ElevationUp,
"elevation_loss": detailedTour.ElevationDown,
"duration": detailedTour.Duration,
"date": detailedTour.Date,
"author": actor,
"trail": trailid,
})
if err := app.Save(summitLogRecord); err != nil {
return "", err
}
}
return trailid, nil
}
func createWaypointsFromTour(app core.App, tour *DetailedKomootTour, user string, trailid string) error {
collection, err := app.FindCollectionByNameOrId("waypoints")
if err != nil {
return err
}
for _, wp := range tour.Embedded.Timeline.Embedded.Items {
photos, err := fetchWaypointPhotos(wp)
if err != nil {
return err
}
record := core.NewRecord(collection)
wpDescription := ""
if len(wp.Embedded.Reference.Embedded.Tips.Embedded.Items) > 0 {
wpDescription = wp.Embedded.Reference.Embedded.Tips.Embedded.Items[0].Text
}
wpLat := wp.Embedded.Reference.StartPoint.Lat
if wpLat == 0 {
wpLat = tour.StartPoint.Lat
}
wpLon := wp.Embedded.Reference.StartPoint.Lng
if wpLon == 0 {
wpLon = tour.StartPoint.Lng
}
record.Load(map[string]any{
"name": wp.Embedded.Reference.Name,
"description": wpDescription,
"lat": wpLat,
"lon": wpLon,
"icon": "circle",
"author": user,
"distance_from_start": 0,
"trail": trailid,
})
if photos != nil {
record.Set("photos", photos)
}
if err := app.Save(record); err != nil {
return err
}
}
return nil
}
func fetchRoutePhotos(k *KomootApi, tour *DetailedKomootTour) ([]*filesystem.File, error) {
url := fmt.Sprintf("https://api.komoot.de/v007/tours/%d/cover_images/", tour.ID)
body, err := sendRequest(url, k.buildHeader())
if err != nil {
return nil, err
}
var data *CoverImages
err = json.Unmarshal(body, &data)
if err != nil {
return nil, err
}
photos := make([]*filesystem.File, 0, len(data.Embedded.Items))
for _, img := range data.Embedded.Items {
photo, err := fetchPhoto(img.Src, "", "")
if err != nil {
return nil, err
}
if strings.HasSuffix(photo.Name, ".gif") {
continue
}
photos = append(photos, photo)
//TODO: komoot photos can have location data. Maybe we should create a waypoint for those photos?
}
return photos, nil
}
func fetchWaypointPhotos(wp Item) ([]*filesystem.File, error) {
photos := make([]*filesystem.File, 0, len(wp.Embedded.Reference.Embedded.Images.Embedded.Items))
for _, img := range wp.Embedded.Reference.Embedded.Images.Embedded.Items {
photo, err := fetchPhoto(img.Src, "", "")
if err != nil {
return nil, err
}
if strings.HasSuffix(photo.Name, ".gif") {
continue
}
photos = append(photos, photo)
}
return photos, nil
}
func fetchPhoto(url string, width string, height string) (*filesystem.File, error) {
url = strings.Replace(url, "{crop}", "false", 1)
url = strings.Replace(url, "{width}", width, 1)
url = strings.Replace(url, "{height}", height, 1)
bytes, err := sendRequest(url, nil)
if err != nil {
return nil, err
}
return filesystem.NewFileFromBytes(bytes, "photo")
}
func generateTourGPX(detailedTour *DetailedKomootTour) (*filesystem.File, error) {
var points []gpx.GPXPoint
for _, item := range detailedTour.Embedded.Coordinates.Items {
t := detailedTour.Date.Unix() + int64(item.T/1000)
points = append(points, gpx.GPXPoint{
Point: gpx.Point{Latitude: item.Lat, Longitude: item.Lng, Elevation: *gpx.NewNullableFloat64(item.Alt)},
Timestamp: time.Unix(t, 0)})
}
gpxData := &gpx.GPX{
Version: "1.1",
Creator: "komoot GPX Exporter",
Tracks: []gpx.GPXTrack{
{
Name: detailedTour.Name,
Segments: []gpx.GPXTrackSegment{
{
Points: points,
},
},
},
},
}
gpxAsXML, err := gpxData.ToXml(gpx.ToXmlParams{Version: "1.1", Indent: true})
if err != nil {
return nil, err
}
gpxFile, err := filesystem.NewFileFromBytes(gpxAsXML, detailedTour.Name+".gpx")
if err != nil {
return nil, err
}
return gpxFile, nil
}

399
db/integrations/komoot/models.go
View File

@@ -1,399 +0,0 @@
package komoot
import (
"time"
"pocketbase/services/trailmerge"
)
type KomootIntegration struct {
Active bool `json:"active"`
Email string `json:"email"`
Password string `json:"password"`
Planned bool `json:"planned"`
Completed bool `json:"completed"`
Privacy string `json:"privacy"`
Merge trailmerge.IntegrationAutoMergeSettings `json:"merge"`
}
type LoginResponse struct {
Email string `json:"email"`
Password string `json:"password"`
User User `json:"user"`
Username string `json:"username"`
}
type Content struct {
HasImage bool `json:"hasImage"`
}
type Fitness struct {
Personalised bool `json:"personalised"`
}
type User struct {
Content Content `json:"content"`
CreatedAt string `json:"createdAt"`
Displayname string `json:"displayname"`
Fitness Fitness `json:"fitness"`
ImageURL string `json:"imageUrl"`
Locale string `json:"locale"`
Metric bool `json:"metric"`
Newsletter bool `json:"newsletter"`
State string `json:"state"`
Username string `json:"username"`
WelcomeMails bool `json:"welcomeMails"`
}
type KomootToursResponse struct {
Embedded Embedded `json:"_embedded"`
Links ResponseLinks `json:"_links"`
Page Page `json:"page"`
}
type StartPoint struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
Alt float64 `json:"alt"`
}
type Surfaces struct {
Type string `json:"type"`
Amount float64 `json:"amount"`
}
type WayTypes struct {
Type string `json:"type"`
Amount float64 `json:"amount"`
}
type Summary struct {
Surfaces []Surfaces `json:"surfaces"`
WayTypes []WayTypes `json:"way_types"`
}
type Difficulty struct {
Grade string `json:"grade"`
ExplanationTechnical string `json:"explanation_technical"`
ExplanationFitness string `json:"explanation_fitness"`
}
type Location struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
}
type Path struct {
Location Location `json:"location"`
Index int `json:"index"`
Reference string `json:"reference,omitempty"`
EndIndex int `json:"end_index,omitempty"`
SegmentType string `json:"segment_type,omitempty"`
}
type Segments struct {
Type string `json:"type"`
From int `json:"from"`
To int `json:"to"`
}
type MapImage struct {
Src string `json:"src"`
Templated bool `json:"templated"`
Type string `json:"type"`
Attribution string `json:"attribution"`
}
type MapImagePreview struct {
Src string `json:"src"`
Templated bool `json:"templated"`
Type string `json:"type"`
Attribution string `json:"attribution"`
}
type VectorMapImage struct {
Src string `json:"src"`
Templated bool `json:"templated"`
Type string `json:"type"`
Attribution string `json:"attribution"`
}
type VectorMapImagePreview struct {
Src string `json:"src"`
Templated bool `json:"templated"`
Type string `json:"type"`
Attribution string `json:"attribution"`
}
type Relation struct {
Href string `json:"href"`
Templated bool `json:"templated"`
}
type CreatorLinks struct {
Relation Relation `json:"relation"`
}
type LinksEmbedded struct {
Creator Creator `json:"creator"`
}
type LinksCreator struct {
Href string `json:"href"`
}
type LinksCoordinates struct {
Href string `json:"href"`
}
type LinksTourLine struct {
Href string `json:"href"`
}
type LinksParticipants struct {
Href string `json:"href"`
}
type LinksWayTypes struct {
Href string `json:"href"`
}
type LinksSurfaces struct {
Href string `json:"href"`
}
type LinksDirections struct {
Href string `json:"href"`
}
type LinksTimeline struct {
Href string `json:"href"`
}
type LinksTranslations struct {
Href string `json:"href"`
}
type LinksCoverImages struct {
Href string `json:"href"`
}
type LinksTourRating struct {
Href string `json:"href"`
}
type TourLinks struct {
Creator LinksCreator `json:"creator"`
Coordinates LinksCoordinates `json:"coordinates"`
TourLine LinksTourLine `json:"tour_line"`
Participants LinksParticipants `json:"participants"`
WayTypes LinksWayTypes `json:"way_types"`
Surfaces LinksSurfaces `json:"surfaces"`
Directions LinksDirections `json:"directions"`
Timeline LinksTimeline `json:"timeline"`
Translations LinksTranslations `json:"translations"`
CoverImages LinksCoverImages `json:"cover_images"`
TourRating LinksTourRating `json:"tour_rating"`
}
type KomootTour struct {
ID int `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Source string `json:"source"`
RoutingVersion string `json:"routing_version"`
Status string `json:"status"`
Date time.Time `json:"date"`
KcalActive int `json:"kcal_active"`
KcalResting int `json:"kcal_resting"`
StartPoint StartPoint `json:"start_point"`
Distance float64 `json:"distance"`
Duration int `json:"duration"`
ElevationUp float64 `json:"elevation_up"`
ElevationDown float64 `json:"elevation_down"`
Sport string `json:"sport"`
Query string `json:"query"`
Constitution int `json:"constitution"`
Summary Summary `json:"summary"`
Difficulty Difficulty `json:"difficulty"`
TourInformation []any `json:"tour_information"`
Path []Path `json:"path"`
Segments []Segments `json:"segments"`
ChangedAt time.Time `json:"changed_at"`
MapImage MapImage `json:"map_image"`
MapImagePreview MapImagePreview `json:"map_image_preview"`
VectorMapImage VectorMapImage `json:"vector_map_image"`
VectorMapImagePreview VectorMapImagePreview `json:"vector_map_image_preview"`
PotentialRouteUpdate bool `json:"potential_route_update"`
Embedded Embedded `json:"_embedded"`
Links TourLinks `json:"_links"`
}
type Embedded struct {
Tours []KomootTour `json:"tours"`
}
type Next struct {
Href string `json:"href"`
}
type ResponseLinks struct {
Next Next `json:"next"`
}
type Page struct {
Size int `json:"size"`
TotalElements int `json:"totalElements"`
TotalPages int `json:"totalPages"`
Number int `json:"number"`
}
type DetailedKomootTour struct {
ID int `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Status string `json:"status"`
Date time.Time `json:"date"`
KcalActive float64 `json:"kcal_active"`
KcalResting float64 `json:"kcal_resting"`
StartPoint StartPoint `json:"start_point"`
Distance float64 `json:"distance"`
Duration int `json:"duration"`
ElevationUp float64 `json:"elevation_up"`
ElevationDown float64 `json:"elevation_down"`
Sport string `json:"sport"`
MapImage MapImage `json:"map_image"`
Difficulty Difficulty `json:"difficulty"`
ChangedAt time.Time `json:"changed_at"`
Embedded DetailedTourEmbedded `json:"_embedded"`
}
type Items struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
Alt float64 `json:"alt"`
T int `json:"t"`
}
type Coordinates struct {
Items []Items `json:"items"`
}
type DetailedTourEmbedded struct {
Coordinates Coordinates `json:"coordinates"`
Timeline Timeline `json:"timeline"`
CoverImages CoverImages `json:"cover_images"`
}
type CoverImages struct {
Embedded CoverImagesEmbedded `json:"_embedded"`
Links Links `json:"_links"`
Page Page `json:"page"`
}
type CoverImagesEmbedded struct {
Items []ImageItem `json:"items"`
}
type Timeline struct {
Embedded TimelineEmbedded `json:"_embedded"`
Links Links `json:"_links"`
Page Page `json:"page"`
}
type TimelineEmbedded struct {
Items []Item `json:"items"`
}
type Item struct {
Index int `json:"index"`
Cover int `json:"cover"`
Type string `json:"type"`
Embedded TimelineItemEmbedded `json:"_embedded"`
}
type TimelineItemEmbedded struct {
Reference Reference `json:"reference"`
}
type Reference struct {
ID int `json:"id"`
Type string `json:"type"`
BaseName string `json:"base_name"`
Name string `json:"name"`
CreatedAt time.Time `json:"created_at"`
ChangedAt time.Time `json:"changed_at"`
Sport string `json:"sport"`
Routable bool `json:"routable"`
StartPoint Point `json:"start_point"`
MidPoint Point `json:"mid_point"`
EndPoint Point `json:"end_point"`
Distance float64 `json:"distance"`
ElevationUp float64 `json:"elevation_up"`
ElevationDown float64 `json:"elevation_down"`
Score float64 `json:"score"`
WikiPOIID string `json:"wiki_poi_id"`
PoorQuality bool `json:"poor_quality"`
Categories []string `json:"categories"`
Flagged bool `json:"flagged"`
Links Links `json:"_links"`
Embedded SubEmbedded `json:"_embedded"`
}
type Point struct {
Lat float64 `json:"lat"`
Lng float64 `json:"lng"`
Alt float64 `json:"alt"`
}
type Links struct {
Self Link `json:"self"`
}
type Link struct {
Href string `json:"href"`
Templated bool `json:"templated,omitempty"`
}
type SubEmbedded struct {
Creator Creator `json:"creator"`
Images Images `json:"images"`
Tips Tips `json:"tips"`
}
type Creator struct {
Username string `json:"username"`
Avatar Avatar `json:"avatar"`
Status string `json:"status"`
Links Links `json:"_links"`
DisplayName string `json:"display_name"`
IsPremium bool `json:"is_premium"`
}
type Avatar struct {
Src string `json:"src"`
Templated bool `json:"templated"`
Type string `json:"type"`
}
type Images struct {
Embedded ImagesEmbedded `json:"_embedded"`
Links Links `json:"_links"`
Page Page `json:"page"`
}
type ImagesEmbedded struct {
Items []ImageItem `json:"items"`
}
type ImageItem struct {
ID int `json:"id"`
Src string `json:"src"`
Rating Rating `json:"rating"`
Templated bool `json:"templated"`
HighlightID int `json:"highlight_id"`
ClientHash string `json:"client_hash,omitempty"`
Location Location `json:"location"`
Type string `json:"type"`
Links Links `json:"_links"`
Embedded SubEmbedded `json:"_embedded"`
}
type Rating struct {
Up int `json:"up"`
Down int `json:"down"`
}
type Tips struct {
Embedded TipsEmbedded `json:"_embedded"`
Links Links `json:"_links"`
Page Page `json:"page"`
}
type TipsEmbedded struct {
Items []TipItem `json:"items"`
}
type TipItem struct {
ID int `json:"id"`
Text string `json:"text"`
Rating Rating `json:"rating"`
CreatedAt time.Time `json:"created_at"`
TextLanguage string `json:"text_language"`
TranslatedText string `json:"translated_text"`
TranslatedTextLanguage string `json:"translated_text_language"`
Attribution string `json:"attribution"`
HighlightID int `json:"highlight_id"`
Links Links `json:"_links"`
Embedded SubEmbedded `json:"_embedded"`
}

386
db/integrations/strava/models.go
View File

@@ -1,386 +0,0 @@
package strava
import (
"time"
"pocketbase/services/trailmerge"
)
type TokenRequest struct {
ClientID int32 `json:"client_id"`
ClientSecret string `json:"client_secret"`
Code string `json:"code"`
GrantType string `json:"grant_type"`
}
type RefreshTokenRequest struct {
ClientID int32 `json:"client_id"`
ClientSecret string `json:"client_secret"`
RefreshToken string `json:"refresh_token"`
GrantType string `json:"grant_type"`
}
type RefreshTokenResponse struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresAt int64 `json:"expires_at"`
}
type StravaIntegration struct {
Active bool `json:"active"`
Routes bool `json:"routes"`
Activities bool `json:"activities"`
ClientID int32 `json:"clientId"`
ClientSecret string `json:"clientSecret"`
AccessToken string `json:"accessToken,omitempty"`
RefreshToken string `json:"refreshToken,omitempty"`
ExpiresAt int64 `json:"expiresAt,omitempty"`
Privacy string `json:"privacy"`
After string `json:"after,omitempty"`
Merge trailmerge.IntegrationAutoMergeSettings `json:"merge"`
}
type StravaRoute struct {
Athlete Athlete `json:"athlete"`
Description string `json:"description"`
Distance float32 `json:"distance"`
ElevationGain float32 `json:"elevation_gain"`
ID int64 `json:"id"`
IDStr string `json:"id_str"`
Map Map `json:"map"`
Name string `json:"name"`
Private bool `json:"private"`
Starred bool `json:"starred"`
Timestamp int `json:"timestamp"`
Type int `json:"type"`
SubType int `json:"sub_type"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
EstimatedMovingTime int `json:"estimated_moving_time"`
Segments []Segments `json:"segments"`
Waypoints []Waypoints `json:"waypoints"`
}
type Athlete struct {
ID int64 `json:"id"`
ResourceState int `json:"resource_state"`
Firstname string `json:"firstname"`
Lastname string `json:"lastname"`
ProfileMedium string `json:"profile_medium"`
Profile string `json:"profile"`
City string `json:"city"`
State string `json:"state"`
Country string `json:"country"`
Sex string `json:"sex"`
Premium bool `json:"premium"`
Summit bool `json:"summit"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
type Map struct {
ID string `json:"id"`
Polyline string `json:"polyline"`
SummaryPolyline string `json:"summary_polyline"`
}
type AthletePrEffort struct {
PrActivityID int64 `json:"pr_activity_id"`
PrElapsedTime int `json:"pr_elapsed_time"`
PrDate time.Time `json:"pr_date"`
EffortCount int `json:"effort_count"`
}
type AthleteSegmentStats struct {
ID int `json:"id"`
ActivityID int `json:"activity_id"`
ElapsedTime int `json:"elapsed_time"`
StartDate time.Time `json:"start_date"`
StartDateLocal time.Time `json:"start_date_local"`
Distance float32 `json:"distance"`
IsKom bool `json:"is_kom"`
}
type Segments struct {
ID int64 `json:"id"`
Name string `json:"name"`
ActivityType string `json:"activity_type"`
Distance float32 `json:"distance"`
AverageGrade float32 `json:"average_grade"`
MaximumGrade float32 `json:"maximum_grade"`
ElevationHigh float32 `json:"elevation_high"`
ElevationLow float32 `json:"elevation_low"`
StartLatlng []float32 `json:"start_latlng"`
EndLatlng []float32 `json:"end_latlng"`
ClimbCategory int `json:"climb_category"`
City string `json:"city"`
State string `json:"state"`
Country string `json:"country"`
Private bool `json:"private"`
AthletePrEffort AthletePrEffort `json:"athlete_pr_effort"`
AthleteSegmentStats AthleteSegmentStats `json:"athlete_segment_stats"`
}
type Waypoints struct {
Latlng []float32 `json:"latlng"`
TargetLatlng []float32 `json:"target_latlng"`
Categories []string `json:"categories"`
Title string `json:"title"`
Description string `json:"description"`
DistanceIntoRoute float64 `json:"distance_into_route"`
}
type StravaActivity struct {
ResourceState int `json:"resource_state"`
Athlete Athlete `json:"athlete"`
Name string `json:"name"`
Distance float64 `json:"distance"`
MovingTime int `json:"moving_time"`
ElapsedTime int `json:"elapsed_time"`
TotalElevationGain float64 `json:"total_elevation_gain"`
Type string `json:"type"`
SportType string `json:"sport_type"`
WorkoutType any `json:"workout_type"`
ID int64 `json:"id"`
ExternalID string `json:"external_id"`
UploadID int64 `json:"upload_id"`
StartDate time.Time `json:"start_date"`
StartDateLocal time.Time `json:"start_date_local"`
Timezone string `json:"timezone"`
StartLatlng any `json:"start_latlng"`
EndLatlng any `json:"end_latlng"`
LocationCity any `json:"location_city"`
LocationState any `json:"location_state"`
LocationCountry string `json:"location_country"`
AchievementCount int `json:"achievement_count"`
KudosCount int `json:"kudos_count"`
CommentCount int `json:"comment_count"`
AthleteCount int `json:"athlete_count"`
PhotoCount int `json:"photo_count"`
Map Map `json:"map"`
Trainer bool `json:"trainer"`
Commute bool `json:"commute"`
Manual bool `json:"manual"`
Private bool `json:"private"`
Flagged bool `json:"flagged"`
GearID string `json:"gear_id"`
FromAcceptedTag bool `json:"from_accepted_tag"`
AverageSpeed float64 `json:"average_speed"`
MaxSpeed float64 `json:"max_speed"`
AverageCadence float64 `json:"average_cadence"`
AverageWatts float64 `json:"average_watts"`
WeightedAverageWatts int `json:"weighted_average_watts"`
Kilojoules float64 `json:"kilojoules"`
DeviceWatts bool `json:"device_watts"`
HasHeartrate bool `json:"has_heartrate"`
AverageHeartrate float64 `json:"average_heartrate"`
MaxHeartrate float64 `json:"max_heartrate"`
MaxWatts int `json:"max_watts"`
PrCount int `json:"pr_count"`
TotalPhotoCount int `json:"total_photo_count"`
HasKudoed bool `json:"has_kudoed"`
SufferScore float64 `json:"suffer_score"`
}
type DetailedStravaActivity struct {
ID int64 `json:"id"`
ResourceState int `json:"resource_state"`
ExternalID string `json:"external_id"`
UploadID int64 `json:"upload_id"`
Athlete Athlete `json:"athlete"`
Name string `json:"name"`
Distance float64 `json:"distance"`
MovingTime int `json:"moving_time"`
ElapsedTime int `json:"elapsed_time"`
TotalElevationGain float64 `json:"total_elevation_gain"`
Type string `json:"type"`
SportType string `json:"sport_type"`
StartDate time.Time `json:"start_date"`
StartDateLocal time.Time `json:"start_date_local"`
Timezone string `json:"timezone"`
StartLatlng []float64 `json:"start_latlng"`
EndLatlng []float64 `json:"end_latlng"`
AchievementCount int `json:"achievement_count"`
KudosCount int `json:"kudos_count"`
CommentCount int `json:"comment_count"`
AthleteCount int `json:"athlete_count"`
PhotoCount int `json:"photo_count"`
Map Map `json:"map"`
Trainer bool `json:"trainer"`
Commute bool `json:"commute"`
Manual bool `json:"manual"`
Private bool `json:"private"`
Flagged bool `json:"flagged"`
GearID string `json:"gear_id"`
FromAcceptedTag bool `json:"from_accepted_tag"`
AverageSpeed float64 `json:"average_speed"`
MaxSpeed float64 `json:"max_speed"`
AverageCadence float64 `json:"average_cadence"`
AverageTemp int `json:"average_temp"`
AverageWatts float64 `json:"average_watts"`
WeightedAverageWatts int `json:"weighted_average_watts"`
Kilojoules float64 `json:"kilojoules"`
DeviceWatts bool `json:"device_watts"`
HasHeartrate bool `json:"has_heartrate"`
MaxWatts int `json:"max_watts"`
ElevHigh float64 `json:"elev_high"`
ElevLow float64 `json:"elev_low"`
PrCount int `json:"pr_count"`
TotalPhotoCount int `json:"total_photo_count"`
HasKudoed bool `json:"has_kudoed"`
WorkoutType int `json:"workout_type"`
SufferScore float64 `json:"suffer_score"`
Description string `json:"description"`
Calories float64 `json:"calories"`
SegmentEfforts []SegmentEfforts `json:"segment_efforts"`
SplitsMetric []SplitsMetric `json:"splits_metric"`
Laps []Laps `json:"laps"`
Gear Gear `json:"gear"`
PartnerBrandTag any `json:"partner_brand_tag"`
Photos Photos `json:"photos"`
HighlightedKudosers []HighlightedKudosers `json:"highlighted_kudosers"`
HideFromHome bool `json:"hide_from_home"`
DeviceName string `json:"device_name"`
EmbedToken string `json:"embed_token"`
SegmentLeaderboardOptOut bool `json:"segment_leaderboard_opt_out"`
LeaderboardOptOut bool `json:"leaderboard_opt_out"`
}
type SegmentActivity struct {
ID int64 `json:"id"`
ResourceState int `json:"resource_state"`
}
type Segment struct {
ID int64 `json:"id"`
ResourceState int `json:"resource_state"`
Name string `json:"name"`
ActivityType string `json:"activity_type"`
Distance float64 `json:"distance"`
AverageGrade float64 `json:"average_grade"`
MaximumGrade float64 `json:"maximum_grade"`
ElevationHigh float64 `json:"elevation_high"`
ElevationLow float64 `json:"elevation_low"`
StartLatlng []float64 `json:"start_latlng"`
EndLatlng []float64 `json:"end_latlng"`
ClimbCategory int `json:"climb_category"`
City string `json:"city"`
State string `json:"state"`
Country string `json:"country"`
Private bool `json:"private"`
Hazardous bool `json:"hazardous"`
Starred bool `json:"starred"`
}
type SegmentEfforts struct {
ID int64 `json:"id"`
ResourceState int `json:"resource_state"`
Name string `json:"name"`
Activity SegmentActivity `json:"activity"`
Athlete Athlete `json:"athlete"`
ElapsedTime int `json:"elapsed_time"`
MovingTime int `json:"moving_time"`
StartDate time.Time `json:"start_date"`
StartDateLocal time.Time `json:"start_date_local"`
Distance float64 `json:"distance"`
StartIndex int `json:"start_index"`
EndIndex int `json:"end_index"`
AverageCadence float64 `json:"average_cadence"`
DeviceWatts bool `json:"device_watts"`
AverageWatts float64 `json:"average_watts"`
Segment Segment `json:"segment"`
KomRank any `json:"kom_rank"`
PrRank any `json:"pr_rank"`
Achievements []any `json:"achievements"`
Hidden bool `json:"hidden"`
}
type SplitsMetric struct {
Distance float64 `json:"distance"`
ElapsedTime int `json:"elapsed_time"`
ElevationDifference float64 `json:"elevation_difference"`
MovingTime int `json:"moving_time"`
Split int `json:"split"`
AverageSpeed float64 `json:"average_speed"`
PaceZone int `json:"pace_zone"`
}
type Laps struct {
ID int64 `json:"id"`
ResourceState int `json:"resource_state"`
Name string `json:"name"`
Activity SegmentActivity `json:"activity"`
Athlete Athlete `json:"athlete"`
ElapsedTime int `json:"elapsed_time"`
MovingTime int `json:"moving_time"`
StartDate time.Time `json:"start_date"`
StartDateLocal time.Time `json:"start_date_local"`
Distance float64 `json:"distance"`
StartIndex int `json:"start_index"`
EndIndex int `json:"end_index"`
TotalElevationGain float64 `json:"total_elevation_gain"`
AverageSpeed float64 `json:"average_speed"`
MaxSpeed float64 `json:"max_speed"`
AverageCadence float64 `json:"average_cadence"`
DeviceWatts bool `json:"device_watts"`
AverageWatts float64 `json:"average_watts"`
LapIndex int `json:"lap_index"`
Split int `json:"split"`
}
type Gear struct {
ID string `json:"id"`
Primary bool `json:"primary"`
Name string `json:"name"`
ResourceState int `json:"resource_state"`
Distance int `json:"distance"`
}
type Urls struct {
Num100 string `json:"100"`
Num600 string `json:"600"`
}
type Primary struct {
ID any `json:"id"`
UniqueID string `json:"unique_id"`
Urls Urls `json:"urls"`
Source int `json:"source"`
}
type Photos struct {
Primary Primary `json:"primary"`
UsePrimaryPhoto bool `json:"use_primary_photo"`
Count int `json:"count"`
}
type HighlightedKudosers struct {
DestinationURL string `json:"destination_url"`
DisplayName string `json:"display_name"`
AvatarURL string `json:"avatar_url"`
ShowName bool `json:"show_name"`
}
type ActivityStreamResponse struct {
LatLng LatLngStream `json:"latlng"`
Altitude AltitudeStream `json:"altitude"`
Time TimeStream `json:"time"`
}
type ActivityStream struct {
OriginalSize int `json:"original_size"`
Resolution string `json:"resolution"`
SeriesType string `json:"series_type"`
}
type TimeStream struct {
ActivityStream
Data []int `json:"data"`
}
type LatLngStream struct {
ActivityStream
Data [][]float64 `json:"data"`
}
type AltitudeStream struct {
ActivityStream
Data []float64 `json:"data"`
}

709
db/integrations/strava/strava.go
View File

@@ -1,709 +0,0 @@
package strava
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/filesystem"
"github.com/pocketbase/pocketbase/tools/security"
"github.com/tkrajina/gpxgo/gpx"
"github.com/twpayne/go-polyline"
"pocketbase/services/trailmerge"
"pocketbase/util"
)
type StravaApi struct {
AceessToken string
}
func SyncStrava(app core.App, client meilisearch.ServiceManager) error {
integrations, err := app.FindAllRecords("integrations", dbx.NewExp("true"))
if err != nil {
return err
}
for _, i := range integrations {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return errors.New("POCKETBASE_ENCRYPTION_KEY not set")
}
userId := i.GetString("user")
actor, err := app.FindFirstRecordByData("activitypub_actors", "user", userId)
if err != nil {
warning := fmt.Sprintf("no actor found for user: %s\n", userId)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
ctx, err := util.GetSafeActorContext(nil, actor)
if err != nil {
continue
}
stravaString := i.GetString("strava")
var stravaIntegration StravaIntegration
err = json.Unmarshal([]byte(stravaString), &stravaIntegration)
if err != nil {
return err
}
if !stravaIntegration.Active || stravaIntegration.RefreshToken == "" {
continue
}
decryptedSecret, err := security.Decrypt(stravaIntegration.ClientSecret, encryptionKey)
if err != nil {
return err
}
decryptedRefreshToken, err := security.Decrypt(stravaIntegration.RefreshToken, encryptionKey)
if err != nil {
return err
}
request := RefreshTokenRequest{
ClientID: stravaIntegration.ClientID,
ClientSecret: string(decryptedSecret),
RefreshToken: string(decryptedRefreshToken),
GrantType: "refresh_token",
}
r, err := GetStravaToken(request)
if err != nil {
warning := fmt.Sprintf("error refreshing strava access token: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
if r.AccessToken != "" {
stravaIntegration.AccessToken = r.AccessToken
}
if r.RefreshToken != "" {
stravaIntegration.RefreshToken = r.RefreshToken
}
if r.AccessToken != "" {
stravaIntegration.ExpiresAt = r.ExpiresAt
}
if stravaIntegration.Routes {
page := 1
hasMore := true
for hasMore {
routes, err := fetchStravaRoutes(r.AccessToken, page)
hasMore = len(routes) > 0
page += 1
if err != nil {
warning := fmt.Sprintf("error fetching routes from strava: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
err = syncTrailsWithRoutes(app, client, ctx, stravaIntegration, r.AccessToken, userId, actor, routes)
if err != nil {
warning := fmt.Sprintf("error syncing strava routes with trails: %v\n", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
}
}
if stravaIntegration.Activities {
page := 1
hasMore := true
for hasMore {
var after int64 = 0
if stravaIntegration.After != "" {
t, err := time.Parse("2006-01-02", stravaIntegration.After)
if err != nil {
return err
}
t = t.UTC()
after = t.Unix()
}
activities, err := fetchStravaActivities(r.AccessToken, page, after)
hasMore = len(activities) > 0
page += 1
if err != nil {
warning := fmt.Sprintf("error fetching activities from strava: %v", err)
fmt.Print(warning)
app.Logger().Warn(warning)
break
}
err = syncTrailsWithActivities(app, client, ctx, stravaIntegration, r.AccessToken, userId, actor, activities)
if err != nil {
warning := fmt.Sprintf("error syncing strava activities with trails: %v", err)
fmt.Print(warning)
app.Logger().Warn(warning)
continue
}
}
}
b, err := json.Marshal(stravaIntegration)
if err != nil {
return err
}
i.Set("strava", string(b))
err = app.Save(i)
if err != nil {
return err
}
}
return nil
}
func GetStravaToken(request any) (*RefreshTokenResponse, error) {
const stravaTokenURL = "https://www.strava.com/oauth/token"
requestBody, err := json.Marshal(request)
if err != nil {
return nil, err
}
req, err := http.NewRequest("POST", stravaTokenURL, bytes.NewBuffer(requestBody))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to get token: received status %d", resp.StatusCode)
}
var tokenResponse RefreshTokenResponse
if err := json.NewDecoder(resp.Body).Decode(&tokenResponse); err != nil {
return nil, err
}
return &tokenResponse, nil
}
func fetchStravaRoutes(accessToken string, page int) ([]StravaRoute, error) {
stravaRoutesURL := fmt.Sprintf("https://www.strava.com/api/v3/athlete/routes?page=%d", page)
req, err := http.NewRequest("GET", stravaRoutesURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to fetch routes: received status %d", resp.StatusCode)
}
var routes []StravaRoute
if err := json.NewDecoder(resp.Body).Decode(&routes); err != nil {
return nil, err
}
return routes, nil
}
func fetchStravaActivities(accessToken string, page int, after int64) ([]StravaActivity, error) {
stravaRoutesURL := fmt.Sprintf("https://www.strava.com/api/v3/athlete/activities?page=%d&after=%d", page, after)
req, err := http.NewRequest("GET", stravaRoutesURL, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to fetch activities: received status %d", resp.StatusCode)
}
var activities []StravaActivity
if err := json.NewDecoder(resp.Body).Decode(&activities); err != nil {
return nil, err
}
return activities, nil
}
func syncTrailsWithRoutes(app core.App, client meilisearch.ServiceManager, ctx context.Context, i StravaIntegration, accessToken string, user string, actor *core.Record, routes []StravaRoute) error {
for _, route := range routes {
existingTrail, err := util.FindTrailByExternalReference(app, "strava", route.IDStr)
if err != nil {
return err
}
if existingTrail != nil {
continue
}
gpx, err := fetchRouteGPX(route, accessToken)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to fetch GPX for route '%s': %v", route.Name, err))
continue
}
trailid, err := createTrailFromRoute(app, route, gpx, user, actor.Id, i.Privacy)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create trail for route '%s': %v", route.Name, err))
continue
}
err = createWaypointsFromRoute(app, route, user, trailid)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create waypoints for route '%s': %v", route.Name, err))
continue
}
if err := trailmerge.TryAutoMergeImportedTrail(app, client, ctx, actor, trailid, i.Merge); err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to auto-merge imported Strava route '%s': %v", route.Name, err))
}
}
return nil
}
func fetchRouteGPX(route StravaRoute, accessToken string) (*filesystem.File, error) {
url := fmt.Sprintf("https://www.strava.com/api/v3/routes/%s/export_gpx", route.IDStr)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer func() {
if resp.Body != nil {
resp.Body.Close()
}
}()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to fetch GPX: received status %d", resp.StatusCode)
}
var buf bytes.Buffer
_, err = io.Copy(&buf, resp.Body)
if err != nil {
return nil, err
}
gpxFile, err := filesystem.NewFileFromBytes(buf.Bytes(), route.Name+".gpx")
if err != nil {
return nil, err
}
return gpxFile, nil
}
func createTrailFromRoute(app core.App, route StravaRoute, gpx *filesystem.File, user string, actor string, privacy string) (string, error) {
trailid := security.RandomStringWithAlphabet(core.DefaultIdLength, core.DefaultIdAlphabet)
collection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return "", err
}
record := core.NewRecord(collection)
buf := []byte(route.Map.SummaryPolyline)
coords, _, _ := polyline.DecodeCoords(buf)
var lat, lon float64
if len(coords) > 0 && len(coords[0]) >= 2 {
lat = coords[0][0]
lon = coords[0][1]
} else {
app.Logger().Warn("Warning: No coordinates available, setting lat/lon to 0")
lat, lon = 0, 0
}
bikeCategory, _ := app.FindFirstRecordByData("categories", "name", "Biking")
hikeCategory, _ := app.FindFirstRecordByData("categories", "name", "Walking")
category := ""
if route.Type == 1 && bikeCategory != nil {
category = bikeCategory.Id
} else if route.Type == 2 && hikeCategory != nil {
category = hikeCategory.Id
}
public := !route.Private
if privacy == "settings" {
privacySettings := struct {
Trails string `json:"trails"`
}{}
settings, _ := app.FindFirstRecordByData("settings", "user", user)
err = settings.UnmarshalJSONField("privacy", &privacySettings)
if err != nil {
return "", err
}
public = privacySettings.Trails == "public"
}
record.Load(map[string]any{
"id": trailid,
"name": route.Name,
"description": route.Description,
"public": public,
"distance": route.Distance,
"elevation_gain": route.ElevationGain,
"duration": route.EstimatedMovingTime,
"date": time.Unix(int64(route.Timestamp), 0),
"lat": lat,
"lon": lon,
"difficulty": "easy",
"category": category,
"author": actor,
})
if gpx != nil {
record.Set("gpx", gpx)
}
if err := app.Save(record); err != nil {
return "", err
}
if err := util.EnsureTrailExternalReference(app, trailid, "strava", route.IDStr); err != nil {
return "", err
}
return trailid, err
}
func createWaypointsFromRoute(app core.App, route StravaRoute, user string, trailid string) error {
collection, err := app.FindCollectionByNameOrId("waypoints")
if err != nil {
return err
}
for i, wp := range route.Waypoints {
record := core.NewRecord(collection)
record.Set("name", strconv.Itoa(i))
record.Set("description", wp.Description)
record.Set("lat", wp.Latlng[0])
record.Set("lon", wp.Latlng[1])
record.Set("icon", "circle")
record.Set("author", user)
record.Set("distance_from_start", wp.DistanceIntoRoute)
record.Set("trail", trailid)
if err := app.Save(record); err != nil {
return err
}
}
return nil
}
func syncTrailsWithActivities(app core.App, client meilisearch.ServiceManager, ctx context.Context, i StravaIntegration, accessToken string, user string, actor *core.Record, activities []StravaActivity) error {
for _, activity := range activities {
existingTrail, err := util.FindTrailByExternalReference(app, "strava", strconv.Itoa(int(activity.ID)))
if err != nil {
return err
}
if existingTrail != nil {
continue
}
detailedActivity, err := fetchDetailedActivity(activity, accessToken)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to fetch detailed activity '%s': %v", activity.Name, err))
continue
}
gpx, err := generateActivityGPX(detailedActivity, accessToken)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to fetch GPX for activity '%s': %v", activity.Name, err))
continue
}
trailID, err := createTrailFromActivity(app, detailedActivity, gpx, user, actor.Id, i.Privacy)
if err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to create trail from activity '%s': %v", activity.Name, err))
continue
}
if err := trailmerge.TryAutoMergeImportedTrail(app, client, ctx, actor, trailID, i.Merge); err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to auto-merge imported Strava activity '%s': %v", activity.Name, err))
}
}
return nil
}
func fetchDetailedActivity(activity StravaActivity, accessToken string) (*DetailedStravaActivity, error) {
url := fmt.Sprintf("https://www.strava.com/api/v3/activities/%d", activity.ID)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to fetch activity: received status %d", resp.StatusCode)
}
var detailedActivity DetailedStravaActivity
if err := json.NewDecoder(resp.Body).Decode(&detailedActivity); err != nil {
return nil, err
}
return &detailedActivity, nil
}
func createTrailFromActivity(app core.App, activity *DetailedStravaActivity, gpx *filesystem.File, user string, actor string, privacy string) (string, error) {
if len(activity.StartLatlng) < 2 {
return "", nil
}
collection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return "", err
}
var photo *filesystem.File
if len(activity.Photos.Primary.Urls.Num600) > 0 {
photo, err = fetchActivityPhoto(activity)
if err != nil {
return "", err
}
}
record := core.NewRecord(collection)
activityMap := map[string]string{
"AlpineSki": "Skiing",
"BackcountrySki": "Skiing",
"Canoeing": "Canoeing",
"Crossfit": "Workout",
"EBikeRide": "Biking",
"Elliptical": "Workout",
"Golf": "Walking",
"Handcycle": "Biking",
"Hike": "Hiking",
"IceSkate": "Skiing",
"InlineSkate": "Biking",
"Kayaking": "Canoeing",
"Kitesurf": "Canoeing",
"NordicSki": "Skiing",
"Ride": "Biking",
"RockClimbing": "Climbing",
"RollerSki": "Skiing",
"Rowing": "Canoeing",
"Run": "Walking",
"Sail": "Canoeing",
"Skateboard": "Walking",
"Snowboard": "Skiing",
"Snowshoe": "Hiking",
"Soccer": "Workout",
"StairStepper": "Workout",
"StandUpPaddling": "Canoeing",
"Surfing": "Canoeing",
"Swim": "Workout",
"Velomobile": "Biking",
"VirtualRide": "Biking",
"VirtualRun": "Walking",
"Walk": "Walking",
"WeightTraining": "Workout",
"Wheelchair": "Walking",
"Windsurf": "Canoeing",
"Workout": "Workout",
"Yoga": "Workout",
}
category, _ := app.FindFirstRecordByData("categories", "name", activityMap[activity.Type])
categoryId := ""
if category != nil {
categoryId = category.Id
}
public := !activity.Private
if privacy == "settings" {
privacySettings := struct {
Trails string `json:"trails"`
}{}
settings, _ := app.FindFirstRecordByData("settings", "user", user)
err = settings.UnmarshalJSONField("privacy", &privacySettings)
if err != nil {
return "", err
}
public = privacySettings.Trails == "public"
}
record.Load(map[string]any{
"name": activity.Name,
"description": activity.Description,
"public": public,
"distance": activity.Distance,
"elevation_gain": activity.TotalElevationGain,
"duration": activity.ElapsedTime,
"date": activity.StartDate,
"lat": activity.StartLatlng[0],
"lon": activity.StartLatlng[1],
"difficulty": "easy",
"category": categoryId,
"author": actor,
})
if photo != nil {
record.Set("photos", photo)
}
if gpx != nil {
record.Set("gpx", gpx)
}
if err := app.Save(record); err != nil {
return "", err
}
if err := util.EnsureTrailExternalReference(app, record.Id, "strava", strconv.Itoa(int(activity.ID))); err != nil {
return "", err
}
return record.Id, nil
}
func fetchActivityPhoto(activity *DetailedStravaActivity) (*filesystem.File, error) {
req, err := http.NewRequest("GET", activity.Photos.Primary.Urls.Num600, nil)
if err != nil {
return nil, err
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to fetch photo: received status %d", resp.StatusCode)
}
var buf bytes.Buffer
_, err = io.Copy(&buf, resp.Body)
if err != nil {
return nil, err
}
photo, err := filesystem.NewFileFromBytes(buf.Bytes(), "photo")
if err != nil {
return nil, err
}
return photo, nil
}
func generateActivityGPX(activity *DetailedStravaActivity, accessToken string) (*filesystem.File, error) {
url := fmt.Sprintf("https://www.strava.com/api/v3/activities/%d/streams?keys=latlng,time,altitude&key_by_type=true", activity.ID)
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("failed to fetch activity: %s", resp.Status)
}
var streamResponse ActivityStreamResponse
if err := json.NewDecoder(resp.Body).Decode(&streamResponse); err != nil {
return nil, err
}
latLngStream := streamResponse.LatLng
timeStream := streamResponse.Time
altitudeStream := streamResponse.Altitude
var points []gpx.GPXPoint
for i, latlng := range latLngStream.Data {
lat := latlng[0]
lon := latlng[1]
alt := altitudeStream.Data[i]
t := activity.StartDate.Unix() + int64(timeStream.Data[i])
points = append(points, gpx.GPXPoint{
Point: gpx.Point{Latitude: lat, Longitude: lon, Elevation: *gpx.NewNullableFloat64(alt)},
Timestamp: time.Unix(t, 0)})
}
gpxData := &gpx.GPX{
Version: "1.1",
Creator: "Strava GPX Exporter",
Tracks: []gpx.GPXTrack{
{
Name: activity.Name,
Segments: []gpx.GPXTrackSegment{
{
Points: points,
},
},
},
},
}
gpxAsXML, err := gpxData.ToXml(gpx.ToXmlParams{Version: "1.1", Indent: true})
if err != nil {
return nil, err
}
gpxFile, err := filesystem.NewFileFromBytes(gpxAsXML, activity.Name+".gpx")
if err != nil {
return nil, err
}
return gpxFile, nil
}

170
db/main.go
View File

@@ -1,12 +1,14 @@
package main
import (
"context"
"fmt"
"log"
"os"
"strings"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/plugins/migratecmd"
@@ -14,9 +16,7 @@ import (
"pocketbase/commands"
"pocketbase/hooks"
"pocketbase/integrations/hammerhead"
"pocketbase/integrations/komoot"
"pocketbase/integrations/strava"
"pocketbase/pluginsystem"
"pocketbase/routes"
_ "pocketbase/migrations"
@@ -55,6 +55,9 @@ func verifySettings(app core.App) {
}
func main() {
if len(os.Args) > 1 && os.Args[1] == "plugin-worker" {
os.Exit(pluginsystem.RunPluginWorker(context.Background(), os.Stdin, os.Stdout, os.Stderr))
}
app := pocketbase.New()
client := initializeMeilisearch()
@@ -89,6 +92,10 @@ func setupEventHandlers(app *pocketbase.PocketBase, client meilisearch.ServiceMa
app.OnRecordAfterCreateSuccess("users").BindFunc(hooks.CreateUserHandler(client))
app.OnRecordAfterUpdateSuccess("users").BindFunc(hooks.UpdateUserHandler(client))
app.OnRecordAfterCreateSuccess("activitypub_actors").BindFunc(hooks.CreateActorHandler(client))
app.OnRecordAfterUpdateSuccess("activitypub_actors").BindFunc(hooks.UpdateActorHandler(client))
app.OnRecordAfterDeleteSuccess("activitypub_actors").BindFunc(hooks.DeleteActorHandler(client))
app.OnRecordAfterCreateSuccess("trails").BindFunc(hooks.CreateTrailHandler(client))
app.OnRecordAfterUpdateSuccess("trails").BindFunc(hooks.UpdateTrailHandler(client))
app.OnRecordAfterDeleteSuccess("trails").BindFunc(hooks.DeleteTrailHandler(client))
@@ -97,6 +104,8 @@ func setupEventHandlers(app *pocketbase.PocketBase, client meilisearch.ServiceMa
app.OnRecordUpdateRequest("summit_logs").BindFunc(hooks.UpdateSummitLogHandler())
app.OnRecordDeleteRequest("summit_logs").BindFunc(hooks.DeleteSummitLogHandler(client))
app.OnRecordCreateRequest("waypoints").BindFunc(hooks.CreateWaypointHandler())
app.OnRecordCreateRequest("comments").BindFunc(hooks.CreateCommentHandler())
app.OnRecordUpdateRequest("comments").BindFunc(hooks.UpdateCommentHandler())
app.OnRecordDeleteRequest("comments").BindFunc(hooks.DeleteCommentHandler(client))
@@ -117,11 +126,12 @@ func setupEventHandlers(app *pocketbase.PocketBase, client meilisearch.ServiceMa
app.OnRecordCreateRequest("follows").BindFunc(hooks.CreateFollowHandler())
app.OnRecordDeleteRequest("follows").BindFunc(hooks.DeleteFollowHandler())
app.OnRecordsListRequest("integrations").BindFunc(hooks.ListIntegrationHandler())
app.OnRecordCreate("integrations").BindFunc(hooks.CreateIntegrationHandler())
app.OnRecordAfterCreateSuccess("integrations").BindFunc(hooks.CreateUpdateIntegrationSuccessHandler())
app.OnRecordUpdate("integrations").BindFunc(hooks.UpdateIntegrationHandler())
app.OnRecordAfterUpdateSuccess("integrations").BindFunc(hooks.CreateUpdateIntegrationSuccessHandler())
app.OnRecordsListRequest("plugin_instances").BindFunc(hooks.ListPluginInstanceHandler())
app.OnRecordViewRequest("plugin_instances").BindFunc(hooks.ViewPluginInstanceHandler())
app.OnRecordCreate("plugin_instances").BindFunc(hooks.CreatePluginInstanceHandler())
app.OnRecordAfterCreateSuccess("plugin_instances").BindFunc(hooks.CreateUpdatePluginInstanceSuccessHandler())
app.OnRecordUpdate("plugin_instances").BindFunc(hooks.UpdatePluginInstanceHandler())
app.OnRecordAfterUpdateSuccess("plugin_instances").BindFunc(hooks.CreateUpdatePluginInstanceSuccessHandler())
app.OnRecordsListRequest("feed", "profile_feed").BindFunc(hooks.ListFeedHandler())
@@ -162,10 +172,14 @@ func registerRoutes(se *core.ServeEvent, client meilisearch.ServiceManager) {
se.Router.GET("/search/token", routes.SearchToken(client))
se.Router.POST("/integration/strava/token", routes.IntegrationStravaToken)
se.Router.POST("/integration/hammerhead/upload", routes.IntegrationHammerheadUpload)
se.Router.GET("/integration/hammerhead/login", routes.IntegrationHammerheadLogin)
se.Router.GET("/integration/komoot/login", routes.IntegrationKommotLogin)
se.Router.GET("/plugins", routes.PluginSystemPluginsList)
se.Router.POST("/plugins/trail-send", routes.PluginSystemTrailSend)
se.Router.POST("/plugins/auth/validate", routes.PluginSystemSessionAuthValidate)
se.Router.POST("/plugins/category-remap/preview", routes.PluginSystemCategoryRemapPreview)
se.Router.POST("/plugins/category-remap/apply", routes.PluginSystemCategoryRemapApply)
se.Router.POST("/plugins/oauth/start", routes.PluginSystemOAuthStart)
se.Router.POST("/plugins/oauth/callback", routes.PluginSystemOAuthCallback)
se.Router.POST("/plugins/oauth/revoke", routes.PluginSystemOAuthRevoke)
se.Router.POST("/activitypub/activity/process", routes.ActivitypubActivityProcess)
se.Router.GET("/activitypub/actor", routes.ActivitypubActor)
@@ -188,22 +202,9 @@ func registerCronJobs(app core.App, client meilisearch.ServiceManager) {
schedule = "0 2 * * *"
}
app.Cron().MustAdd("integrations", schedule, func() {
err := strava.SyncStrava(app, client)
if err != nil {
warning := fmt.Sprintf("Error syncing with strava: %v", err)
fmt.Println(warning)
app.Logger().Error(warning)
}
err = komoot.SyncKomoot(app, client)
if err != nil {
warning := fmt.Sprintf("Error syncing with komoot: %v", err)
fmt.Println(warning)
app.Logger().Error(warning)
}
err = hammerhead.SyncHammerhead(app, client)
if err != nil {
warning := fmt.Sprintf("Error syncing with hammerhead: %v", err)
app.Cron().MustAdd("plugin-sync", schedule, func() {
if err := routes.PluginSystemSyncConfigured(context.Background(), app, client); err != nil {
warning := fmt.Sprintf("Error syncing with WASM plugins: %v", err)
fmt.Println(warning)
app.Logger().Error(warning)
}
@@ -212,11 +213,66 @@ func registerCronJobs(app core.App, client meilisearch.ServiceManager) {
func initData(app core.App, client meilisearch.ServiceManager) error {
initCategories(app)
initPlugins(app)
initMeilisearchConfig(client)
go initMeilisearchDocuments(app, client)
go func() {
backfillPolylines(app)
initMeilisearchDocuments(app, client)
}()
return nil
}
func initPlugins(app core.App) {
manager := pluginsystem.NewManager(app, "")
if err := manager.SyncInstalledPlugins(context.Background()); err != nil {
warning := fmt.Sprintf("Error discovering WASM plugins: %v", err)
fmt.Println(warning)
app.Logger().Error(warning)
}
}
func backfillPolylines(app core.App) {
const pageSize int64 = 100
var lastID string
var processed int
var failed int
log.Printf("backfill polyline started")
defer func() {
log.Printf("backfill polyline completed: processed=%d failed=%d", processed, failed)
}()
for {
trails := []*core.Record{}
query := app.RecordQuery("trails").
AndWhere(dbx.NewExp("(polyline IS NULL OR polyline = '') AND gpx != ''")).
OrderBy("id ASC").
Limit(pageSize)
if lastID != "" {
query = query.AndWhere(dbx.NewExp("id > {:lastID}", dbx.Params{"lastID": lastID}))
}
err := query.All(&trails)
if err != nil {
log.Printf("backfill polyline query failed after trail %q: %v", lastID, err)
break
}
if len(trails) == 0 {
break
}
for _, r := range trails {
if err := util.SavePolyline(app, r); err != nil {
failed++
log.Printf("backfill polyline failed for trail %s (%q), gpx=%q: %v", r.Id, r.GetString("name"), r.GetString("gpx"), err)
} else {
processed++
}
lastID = r.Id
}
}
}
func initCategories(app core.App) error {
query := app.RecordQuery("categories")
records := []*core.Record{}
@@ -224,10 +280,16 @@ func initCategories(app core.App) error {
if err := query.All(&records); err != nil {
return err
}
if len(records) == 0 {
collection, _ := app.FindCollectionByNameOrId("categories")
if len(records) != 0 {
return nil
}
categories := []string{"Hiking", "Walking", "Climbing", "Skiing", "Canoeing", "Biking"}
collection, err := app.FindCollectionByNameOrId("categories")
if err != nil {
return err
}
categories := []string{"Hiking", "Walking", "Climbing", "Skiing", "Canoeing", "Biking", "Other"}
for _, element := range categories {
record := core.NewRecord(collection)
record.Set("name", element)
@@ -235,12 +297,11 @@ func initCategories(app core.App) error {
"wp_merge_enabled": true,
"wp_merge_radius": 50,
})
f, _ := filesystem.NewFileFromPath("migrations/initial_data/" + strings.ToLower(element) + ".jpg")
if f, err := filesystem.NewFileFromPath("migrations/initial_data/" + strings.ToLower(element) + ".jpg"); err == nil {
record.Set("img", f)
err := app.Save(record)
if err != nil {
return err
}
if err := app.Save(record); err != nil {
return err
}
}
return nil
@@ -251,9 +312,9 @@ func initMeilisearchConfig(client meilisearch.ServiceManager) {
"trails": {
SearchableAttributes: []string{"author_name", "name", "description", "location", "tags"},
FilterableAttributes: []string{
"_geo", "author", "category", "completed", "date", "difficulty",
"id", "_geo", "author", "category", "completed", "date", "difficulty",
"distance", "elevation_gain", "elevation_loss", "likes", "public",
"shares", "tags",
"shares", "tags", "min_lat", "max_lat", "min_lon", "max_lon", "bounding_box_diagonal",
},
SortableAttributes: []string{
"author", "created", "date", "difficulty", "distance",
@@ -267,6 +328,12 @@ func initMeilisearchConfig(client meilisearch.ServiceManager) {
SortableAttributes: []string{"created", "name"},
RankingRules: []string{"words", "typo", "proximity", "attribute", "sort", "exactness"},
},
"actors": {
SearchableAttributes: []string{"username", "preferred_username", "domain"},
FilterableAttributes: []string{"id"},
SortableAttributes: []string{},
RankingRules: []string{"words", "typo", "proximity", "attribute", "sort", "exactness"},
},
}
for indexName, settings := range configs {
@@ -356,5 +423,32 @@ func initMeilisearchDocuments(app core.App, client meilisearch.ServiceManager) e
page++
}
// --- Actors ---
if _, err := client.Index("actors").DeleteAllDocuments(nil); err != nil {
return err
}
page = 0
for {
actors := []*core.Record{}
err := app.RecordQuery("activitypub_actors").
Limit(pageSize).
Offset(page * pageSize).
All(&actors)
if err != nil {
return err
}
if len(actors) == 0 {
break
}
if err := util.IndexActors(actors, client); err != nil {
app.Logger().Warn(fmt.Sprintf("Unable to index actor page %d: %v", page, err))
continue
}
page++
}
return nil
}

35
db/migrations/1748000000_updated_trails_polyline.go Normal file
View File

@@ -0,0 +1,35 @@
package migrations
import (
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
func init() {
m.Register(func(app core.App) error {
collection, err := app.FindCollectionByNameOrId("e864strfxo14pm4")
if err != nil {
return err
}
if collection.Fields.GetByName("polyline") != nil {
return nil
}
field := &core.TextField{}
field.Name = "polyline"
field.Required = false
collection.Fields.Add(field)
return app.Save(collection)
}, func(app core.App) error {
collection, err := app.FindCollectionByNameOrId("e864strfxo14pm4")
if err != nil {
return err
}
collection.Fields.RemoveByName("polyline")
return app.Save(collection)
})
}

36
db/migrations/1778583700_updated_trails_polyline_max.go Normal file
View File

@@ -0,0 +1,36 @@
package migrations
import (
"pocketbase/util"
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
func init() {
m.Register(func(app core.App) error {
collection, err := app.FindCollectionByNameOrId("e864strfxo14pm4")
if err != nil {
return err
}
field, ok := collection.Fields.GetByName("polyline").(*core.TextField)
if ok {
field.Max = util.PolylineMaxLength
}
return app.Save(collection)
}, func(app core.App) error {
collection, err := app.FindCollectionByNameOrId("e864strfxo14pm4")
if err != nil {
return err
}
field, ok := collection.Fields.GetByName("polyline").(*core.TextField)
if ok {
field.Max = 0
}
return app.Save(collection)
})
}

218
db/migrations/1778583800_persist_trail_bounds.go Normal file
View File

@@ -0,0 +1,218 @@
package migrations
import (
"encoding/json"
"pocketbase/util"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
const trailBoundsViewQuery = `SELECT
a.id, a.user,
COALESCE(MAX(t.max_lat), 0) AS max_lat,
COALESCE(MAX(t.max_lon), 0) AS max_lon,
COALESCE(MIN(t.min_lat), 0) AS min_lat,
COALESCE(MIN(t.min_lon), 0) AS min_lon
FROM activitypub_actors a
LEFT JOIN (
SELECT author AS actor_id,
MAX(max_lat) AS max_lat,
MAX(max_lon) AS max_lon,
MIN(min_lat) AS min_lat,
MIN(min_lon) AS min_lon
FROM trails
GROUP BY author
UNION ALL
SELECT ts.actor AS actor_id,
MAX(t.max_lat) AS max_lat,
MAX(t.max_lon) AS max_lon,
MIN(t.min_lat) AS min_lat,
MIN(t.min_lon) AS min_lon
FROM trail_share ts
JOIN trails t ON t.id = ts.trail
GROUP BY ts.actor
UNION ALL
SELECT a2.id AS actor_id,
p.max_lat, p.max_lon, p.min_lat, p.min_lon
FROM activitypub_actors a2
CROSS JOIN (
SELECT
MAX(max_lat) AS max_lat,
MAX(max_lon) AS max_lon,
MIN(min_lat) AS min_lat,
MIN(min_lon) AS min_lon
FROM trails
WHERE public = TRUE
) p
) t ON t.actor_id = a.id
WHERE a.user != ""
GROUP BY a.id;`
const trailStartPointBoundsViewQuery = `SELECT
a.id, a.user,
COALESCE(MAX(t.max_lat), 0) AS max_lat,
COALESCE(MAX(t.max_lon), 0) AS max_lon,
COALESCE(MIN(t.min_lat), 0) AS min_lat,
COALESCE(MIN(t.min_lon), 0) AS min_lon
FROM activitypub_actors a
LEFT JOIN (
SELECT author AS actor_id,
MAX(lat) AS max_lat,
MAX(lon) AS max_lon,
MIN(lat) AS min_lat,
MIN(lon) AS min_lon
FROM trails
GROUP BY author
UNION ALL
SELECT ts.actor AS actor_id,
MAX(t.lat) AS max_lat,
MAX(t.lon) AS max_lon,
MIN(t.lat) AS min_lat,
MIN(t.lon) AS min_lon
FROM trail_share ts
JOIN trails t ON t.id = ts.trail
GROUP BY ts.actor
UNION ALL
SELECT a2.id AS actor_id,
p.max_lat, p.max_lon, p.min_lat, p.min_lon
FROM activitypub_actors a2
CROSS JOIN (
SELECT
MAX(lat) AS max_lat,
MAX(lon) AS max_lon,
MIN(lat) AS min_lat,
MIN(lon) AS min_lon
FROM trails
WHERE public = TRUE
) p
) t ON t.actor_id = a.id
WHERE a.user != ""
GROUP BY a.id;`
func init() {
m.Register(func(app core.App) error {
trailsCollection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return err
}
addTrailBoundsField(trailsCollection, "min_lat")
addTrailBoundsField(trailsCollection, "max_lat")
addTrailBoundsField(trailsCollection, "min_lon")
addTrailBoundsField(trailsCollection, "max_lon")
addTrailBoundsField(trailsCollection, "bounding_box_diagonal")
if err := app.Save(trailsCollection); err != nil {
return err
}
if err := backfillTrailBounds(app); err != nil {
return err
}
boundingBoxCollection, err := app.FindCollectionByNameOrId("trails_bounding_box")
if err != nil {
return err
}
if err := json.Unmarshal([]byte(`{"viewQuery":`+strconvQuote(trailBoundsViewQuery)+`}`), &boundingBoxCollection); err != nil {
return err
}
if err := app.Save(boundingBoxCollection); err != nil {
return err
}
return nil
}, func(app core.App) error {
boundingBoxCollection, err := app.FindCollectionByNameOrId("trails_bounding_box")
if err != nil {
return err
}
if err := json.Unmarshal([]byte(`{"viewQuery":`+strconvQuote(trailStartPointBoundsViewQuery)+`}`), &boundingBoxCollection); err != nil {
return err
}
if err := app.Save(boundingBoxCollection); err != nil {
return err
}
trailsCollection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return err
}
trailsCollection.Fields.RemoveByName("min_lat")
trailsCollection.Fields.RemoveByName("max_lat")
trailsCollection.Fields.RemoveByName("min_lon")
trailsCollection.Fields.RemoveByName("max_lon")
trailsCollection.Fields.RemoveByName("bounding_box_diagonal")
if err := app.Save(trailsCollection); err != nil {
return err
}
return nil
})
}
func addTrailBoundsField(collection *core.Collection, name string) {
if collection.Fields.GetByName(name) != nil {
return
}
collection.Fields.Add(&core.NumberField{
Name: name,
})
}
func backfillTrailBounds(app core.App) error {
const pageSize int64 = 50
lastID := ""
for {
trails := []*core.Record{}
query := app.RecordQuery("trails").
OrderBy("id ASC").
Limit(pageSize)
if lastID != "" {
query = query.AndWhere(dbx.NewExp("id > {:lastID}", dbx.Params{"lastID": lastID}))
}
if err := query.All(&trails); err != nil {
return err
}
if len(trails) == 0 {
return nil
}
for _, trail := range trails {
if err := util.SavePolyline(app, trail); err != nil {
if err := saveDefaultTrailBounds(app, trail); err != nil {
return err
}
}
lastID = trail.Id
}
}
}
func saveDefaultTrailBounds(app core.App, trail *core.Record) error {
lat := trail.GetFloat("lat")
lon := trail.GetFloat("lon")
trail.Set("min_lat", lat)
trail.Set("max_lat", lat)
trail.Set("min_lon", lon)
trail.Set("max_lon", lon)
trail.Set("bounding_box_diagonal", 0)
return app.UnsafeWithoutHooks().Save(trail)
}
func strconvQuote(value string) string {
raw, _ := json.Marshal(value)
return string(raw)
}

536
db/migrations/1780000002_plugin_instances.go Normal file
View File

@@ -0,0 +1,536 @@
package migrations
import (
"encoding/json"
"fmt"
"strconv"
"strings"
"time"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
"github.com/pocketbase/pocketbase/tools/security"
"github.com/pocketbase/pocketbase/tools/types"
)
func init() {
m.Register(func(app core.App) error {
// Create plugin_instances collection
jsonData := `{
"createRule": "@request.auth.id = user.id",
"deleteRule": "@request.auth.id = user.id",
"fields": [
{
"autogeneratePattern": "[a-z0-9]{15}",
"hidden": false,
"id": "text430001001",
"max": 15,
"min": 15,
"name": "id",
"pattern": "^[a-z0-9]+$",
"presentable": false,
"primaryKey": true,
"required": true,
"system": true,
"type": "text"
},
{
"cascadeDelete": true,
"collectionId": "_pb_users_auth_",
"hidden": false,
"id": "relation430001002",
"maxSelect": 1,
"minSelect": 0,
"name": "user",
"presentable": false,
"required": true,
"system": false,
"type": "relation"
},
{
"autogeneratePattern": "",
"hidden": false,
"id": "text430001003",
"max": 64,
"min": 1,
"name": "plugin_id",
"pattern": "^[a-z0-9][a-z0-9_-]*$",
"presentable": false,
"primaryKey": false,
"required": true,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "bool430001004",
"name": "enabled",
"presentable": false,
"required": false,
"system": false,
"type": "bool"
},
{
"hidden": false,
"id": "json430001005",
"maxSize": 2000000,
"name": "auth",
"presentable": false,
"required": false,
"system": false,
"type": "json"
},
{
"hidden": false,
"id": "json430001006",
"maxSize": 2000000,
"name": "config",
"presentable": false,
"required": false,
"system": false,
"type": "json"
},
{
"hidden": false,
"id": "json430001007",
"maxSize": 2000000,
"name": "state",
"presentable": false,
"required": false,
"system": false,
"type": "json"
},
{
"hidden": false,
"id": "select430001008",
"maxSelect": 1,
"name": "status",
"presentable": false,
"required": true,
"system": false,
"type": "select",
"values": [
"configured",
"needs_auth",
"needs_reauth",
"syncing",
"rate_limited",
"unavailable",
"unsupported_protocol",
"error",
"disabled"
]
},
{
"hidden": false,
"id": "json430001009",
"maxSize": 2000000,
"name": "last_error",
"presentable": false,
"required": false,
"system": false,
"type": "json"
},
{
"hidden": false,
"id": "date430001010",
"max": "",
"min": "",
"name": "last_sync_at",
"presentable": false,
"required": false,
"system": false,
"type": "date"
},
{
"hidden": false,
"id": "date430001011",
"max": "",
"min": "",
"name": "retry_not_before",
"presentable": false,
"required": false,
"system": false,
"type": "date"
},
{
"hidden": false,
"id": "autodate430001012",
"name": "created",
"onCreate": true,
"onUpdate": false,
"presentable": false,
"system": false,
"type": "autodate"
},
{
"hidden": false,
"id": "autodate430001013",
"name": "updated",
"onCreate": true,
"onUpdate": true,
"presentable": false,
"system": false,
"type": "autodate"
}
],
"id": "pbc_430001000",
"indexes": [
"CREATE UNIQUE INDEX ` + "`" + `idx_plugin_instances_user_plugin_id` + "`" + ` ON ` + "`" + `plugin_instances` + "`" + ` (` + "`" + `user` + "`" + `, ` + "`" + `plugin_id` + "`" + `)"
],
"listRule": "@request.auth.id = user.id",
"name": "plugin_instances",
"system": false,
"type": "base",
"updateRule": "@request.auth.id = user.id",
"viewRule": "@request.auth.id = user.id"
}`
if _, err := app.FindCollectionByNameOrId("pbc_430001000"); err != nil {
collection := &core.Collection{}
if err := json.Unmarshal([]byte(jsonData), collection); err != nil {
return err
}
if err := app.Save(collection); err != nil {
return err
}
}
if err := migrateLegacyIntegrationsToPluginInstances(app); err != nil {
return err
}
// Remove the previous hard-coded provider settings collection after
// migrating its configuration into plugin_instances. The migration is
// data-only and does not require the corresponding plugin bundles to be
// installed.
if legacyCollection, err := app.FindCollectionByNameOrId("integrations"); err == nil {
if err := app.Delete(legacyCollection); err != nil {
return err
}
}
// Add user field to trail_external_reference and update index to be user-scoped
refCollection, err := app.FindCollectionByNameOrId("trail_external_reference")
if err != nil {
return err
}
if refCollection.Fields.GetByName("user") == nil {
if err := refCollection.Fields.AddMarshaledJSONAt(2, []byte(`{
"cascadeDelete": true,
"collectionId": "_pb_users_auth_",
"hidden": false,
"id": "relation430002001",
"maxSelect": 1,
"minSelect": 0,
"name": "user",
"presentable": false,
"required": false,
"system": false,
"type": "relation"
}`)); err != nil {
return err
}
// Replace the global unique (provider, external_id) index with a
// user-scoped one so the same external trail can be imported by
// multiple users. This must be managed via the collection metadata
// (not a raw DROP INDEX), otherwise app.Save would recreate the old
// index from the still-present metadata entry.
keptIndexes := refCollection.Indexes[:0]
for _, idx := range refCollection.Indexes {
if strings.Contains(idx, "idx_trail_external_reference_provider_external_id") {
continue
}
keptIndexes = append(keptIndexes, idx)
}
refCollection.Indexes = append(keptIndexes,
"CREATE UNIQUE INDEX `idx_trail_external_reference_user_provider_external_id` ON `trail_external_reference` (`user`, `provider`, `external_id`)",
)
if err := app.Save(refCollection); err != nil {
return err
}
refs, err := app.FindAllRecords("trail_external_reference")
if err != nil {
return err
}
for _, ref := range refs {
trailID := ref.GetString("trail")
if trailID == "" {
continue
}
trail, err := app.FindRecordById("trails", trailID)
if err != nil {
continue
}
actor, err := app.FindRecordById("activitypub_actors", trail.GetString("author"))
if err != nil {
continue
}
userID := actor.GetString("user")
if userID == "" {
continue
}
ref.Set("user", userID)
if err := app.Save(ref); err != nil {
return err
}
}
}
return nil
}, nil)
}
func migrateLegacyIntegrationsToPluginInstances(app core.App) error {
if _, err := app.FindCollectionByNameOrId("integrations"); err != nil {
return nil
}
records, err := app.FindAllRecords("integrations")
if err != nil {
return err
}
for _, record := range records {
userID := record.GetString("user")
if userID == "" {
continue
}
if raw := legacyJSONObject(record.GetString("strava")); legacyHasValue(raw["clientId"]) {
auth := legacyPick(raw, "clientId", "clientSecret", "accessToken", "refreshToken", "expiresAt", "tokenType", "scope")
legacyNormalizeStravaAuth(auth)
hostConfig := legacyPick(raw, "privacy", "merge")
hostConfig["planned"] = legacyBool(raw["routes"])
hostConfig["completed"] = legacyBool(raw["activities"])
config := legacyNamespacedPluginConfig(
legacyPick(raw, "after"),
hostConfig,
)
if err := saveLegacyMappedPluginInstance(app, userID, "strava", auth, config, raw); err != nil {
return err
}
}
if raw := legacyJSONObject(record.GetString("komoot")); legacyHasValue(raw["email"]) {
auth := legacyPick(raw, "email", "password")
config := legacyNamespacedPluginConfig(
legacyPick(raw, "after"),
legacyPick(raw, "planned", "completed", "privacy", "merge"),
)
if err := saveLegacyMappedPluginInstance(app, userID, "komoot", auth, config, raw); err != nil {
return err
}
}
if raw := legacyJSONObject(record.GetString("hammerhead")); legacyHasValue(raw["email"]) {
auth := legacyPick(raw, "email", "password")
config := legacyNamespacedPluginConfig(
legacyPick(raw, "after"),
legacyPick(raw, "planned", "completed", "privacy", "merge"),
)
if err := saveLegacyMappedPluginInstance(app, userID, "hammerhead", auth, config, raw); err != nil {
return err
}
}
}
return nil
}
func legacyNamespacedPluginConfig(pluginConfig map[string]any, hostConfig map[string]any) map[string]any {
return map[string]any{
"plugin": nilMap(pluginConfig),
"host": nilMap(hostConfig),
}
}
func saveLegacyMappedPluginInstance(app core.App, userID string, pluginID string, auth map[string]any, config map[string]any, raw map[string]any) error {
enabled := legacyBool(raw["active"]) && legacyPluginAuthComplete(pluginID, auth)
return saveLegacyPluginInstance(app, legacyPluginInstance{
UserID: userID,
PluginID: pluginID,
Enabled: enabled,
Auth: auth,
Config: config,
State: map[string]any{},
Status: legacyPluginInstanceStatus(pluginID, auth, enabled, ""),
LastError: map[string]any{},
})
}
type legacyPluginInstance struct {
UserID string
PluginID string
Enabled bool
Auth map[string]any
Config map[string]any
State map[string]any
Status string
LastError map[string]any
LastSyncAt string
RetryNotBefore string
}
func saveLegacyPluginInstance(app core.App, instance legacyPluginInstance) error {
if instance.UserID == "" || instance.PluginID == "" {
return nil
}
existing, _ := app.FindFirstRecordByFilter(
"plugin_instances",
"user={:user} && plugin_id={:plugin_id}",
dbx.Params{"user": instance.UserID, "plugin_id": instance.PluginID},
)
if existing != nil {
return nil
}
authJSON, err := json.Marshal(nilMap(instance.Auth))
if err != nil {
return err
}
configJSON, err := json.Marshal(nilMap(instance.Config))
if err != nil {
return err
}
stateJSON, err := json.Marshal(nilMap(instance.State))
if err != nil {
return err
}
lastErrorJSON, err := json.Marshal(nilMap(instance.LastError))
if err != nil {
return err
}
status := instance.Status
if status == "" {
status = legacyPluginInstanceStatus(instance.PluginID, instance.Auth, instance.Enabled, "")
}
now := types.NowDateTime().String()
_, err = app.DB().Insert("plugin_instances", dbx.Params{
"id": security.RandomStringWithAlphabet(15, "abcdefghijklmnopqrstuvwxyz0123456789"),
"user": instance.UserID,
"plugin_id": instance.PluginID,
"enabled": instance.Enabled,
"auth": string(authJSON),
"config": string(configJSON),
"state": string(stateJSON),
"status": status,
"last_error": string(lastErrorJSON),
"last_sync_at": instance.LastSyncAt,
"retry_not_before": instance.RetryNotBefore,
"created": now,
"updated": now,
}).Execute()
return err
}
func legacyPluginInstanceStatus(pluginID string, auth map[string]any, enabled bool, previous string) string {
if !legacyPluginAuthComplete(pluginID, auth) {
return "needs_auth"
}
if !enabled {
return "disabled"
}
switch previous {
case "configured", "needs_reauth", "syncing", "rate_limited", "unavailable", "unsupported_protocol", "error":
return previous
default:
return "configured"
}
}
func legacyPluginAuthComplete(pluginID string, auth map[string]any) bool {
switch pluginID {
case "strava":
return legacyHasValue(auth["clientId"]) && legacyHasValue(auth["clientSecret"]) && legacyHasValue(auth["refreshToken"])
case "komoot", "hammerhead":
return legacyHasValue(auth["email"]) && legacyHasValue(auth["password"])
default:
return false
}
}
func legacyNormalizeStravaAuth(auth map[string]any) {
legacyStringAuthFields(auth, "clientId", "clientSecret", "accessToken", "refreshToken", "tokenType", "scope")
switch value := auth["expiresAt"].(type) {
case float64:
if value > 0 {
auth["expiresAt"] = time.Unix(int64(value), 0).UTC().Format(time.RFC3339)
}
case int64:
if value > 0 {
auth["expiresAt"] = time.Unix(value, 0).UTC().Format(time.RFC3339)
}
case int:
if value > 0 {
auth["expiresAt"] = time.Unix(int64(value), 0).UTC().Format(time.RFC3339)
}
}
}
func legacyStringAuthFields(auth map[string]any, keys ...string) {
for _, key := range keys {
switch value := auth[key].(type) {
case string:
// already normalized
case float64:
auth[key] = strconv.FormatFloat(value, 'f', -1, 64)
case int64:
auth[key] = strconv.FormatInt(value, 10)
case int:
auth[key] = strconv.Itoa(value)
case nil:
// leave absent/null values untouched so completeness checks still fail
default:
auth[key] = fmt.Sprint(value)
}
}
}
func legacyJSONObject(raw string) map[string]any {
if raw == "" {
return map[string]any{}
}
var data map[string]any
if err := json.Unmarshal([]byte(raw), &data); err != nil || data == nil {
return map[string]any{}
}
return data
}
func legacyPick(src map[string]any, keys ...string) map[string]any {
out := map[string]any{}
for _, key := range keys {
if value, ok := src[key]; ok && value != nil {
out[key] = value
}
}
return out
}
func legacyHasValue(value any) bool {
switch v := value.(type) {
case nil:
return false
case string:
return strings.TrimSpace(v) != ""
default:
return true
}
}
func legacyBool(value any) bool {
b, _ := value.(bool)
return b
}
func nilMap(value map[string]any) map[string]any {
if value == nil {
return map[string]any{}
}
return value
}

198
db/migrations/1780000004_plugin_system.go Normal file
View File

@@ -0,0 +1,198 @@
package migrations
import (
"encoding/json"
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
func init() {
m.Register(func(app core.App) error {
return createInstalledPluginsCollection(app)
}, func(app core.App) error {
if collection, err := app.FindCollectionByNameOrId("installed_plugins"); err == nil {
if err := app.Delete(collection); err != nil {
return err
}
}
return nil
})
}
func createInstalledPluginsCollection(app core.App) error {
if _, err := app.FindCollectionByNameOrId("installed_plugins"); err == nil {
return nil
}
jsonData := `{
"createRule": null,
"deleteRule": null,
"fields": [
{
"autogeneratePattern": "[a-z0-9]{15}",
"hidden": false,
"id": "textplginsid01",
"max": 15,
"min": 15,
"name": "id",
"pattern": "^[a-z0-9]+$",
"presentable": false,
"primaryKey": true,
"required": true,
"system": true,
"type": "text"
},
{
"hidden": false,
"id": "textplginpid1",
"max": 128,
"min": 1,
"name": "plugin_id",
"pattern": "^[a-z0-9][a-z0-9_-]*$",
"presentable": false,
"required": true,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "textplginname",
"max": 256,
"min": 1,
"name": "name",
"pattern": "",
"presentable": true,
"required": true,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "selectplgtype",
"maxSelect": 1,
"name": "type",
"presentable": false,
"required": true,
"system": false,
"type": "select",
"values": ["trails"]
},
{
"hidden": false,
"id": "textplginvers",
"max": 64,
"min": 1,
"name": "version",
"pattern": "",
"presentable": false,
"required": true,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "textplginrunt",
"max": 32,
"min": 1,
"name": "runtime",
"pattern": "",
"presentable": false,
"required": true,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "textplginpath",
"max": 0,
"min": 0,
"name": "path",
"pattern": "",
"presentable": false,
"required": false,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "jsonplginman",
"maxSize": 2000000,
"name": "manifest",
"presentable": false,
"required": true,
"system": false,
"type": "json"
},
{
"hidden": false,
"id": "jsonplgincfg",
"maxSize": 2000000,
"name": "config",
"presentable": false,
"required": false,
"system": false,
"type": "json"
},
{
"hidden": false,
"id": "selectplginst",
"maxSelect": 1,
"name": "status",
"presentable": false,
"required": true,
"system": false,
"type": "select",
"values": ["available", "disabled", "error"]
},
{
"hidden": false,
"id": "textplginerr",
"max": 0,
"min": 0,
"name": "error",
"pattern": "",
"presentable": false,
"required": false,
"system": false,
"type": "text"
},
{
"hidden": false,
"id": "autoplgcreate",
"name": "created",
"onCreate": true,
"onUpdate": false,
"presentable": false,
"system": false,
"type": "autodate"
},
{
"hidden": false,
"id": "autoplgupdate",
"name": "updated",
"onCreate": true,
"onUpdate": true,
"presentable": false,
"system": false,
"type": "autodate"
}
],
"id": "pbc_430002000",
"indexes": [
"CREATE UNIQUE INDEX ` + "`" + `idx_installed_plugins_plugin_id` + "`" + ` ON ` + "`" + `installed_plugins` + "`" + ` (` + "`" + `plugin_id` + "`" + `)"
],
"listRule": null,
"name": "installed_plugins",
"system": false,
"type": "base",
"updateRule": null,
"viewRule": null
}`
collection := &core.Collection{}
if err := json.Unmarshal([]byte(jsonData), collection); err != nil {
return err
}
return app.Save(collection)
}

46
db/migrations/1780000005_add_other_category.go Normal file
View File

@@ -0,0 +1,46 @@
package migrations
import (
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
"github.com/pocketbase/pocketbase/tools/filesystem"
)
func init() {
m.Register(func(app core.App) error {
categories, err := app.FindAllRecords("categories")
if err != nil {
return err
}
if len(categories) == 0 {
return nil
}
existing, _ := app.FindFirstRecordByData("categories", "name", "Other")
if existing != nil {
return nil
}
collection, err := app.FindCollectionByNameOrId("categories")
if err != nil {
return err
}
record := core.NewRecord(collection)
record.Set("name", "Other")
record.Set("settings", map[string]any{
"wp_merge_enabled": true,
"wp_merge_radius": 50,
})
if file, err := filesystem.NewFileFromPath("migrations/initial_data/other.jpg"); err == nil {
record.Set("img", file)
}
return app.Save(record)
}, func(app core.App) error {
record, _ := app.FindFirstRecordByData("categories", "name", "Other")
if record == nil {
return nil
}
return app.Delete(record)
})
}

262
db/migrations/1780000006_trail_external_reference_provider_text.go Normal file
View File

@@ -0,0 +1,262 @@
package migrations
import (
"strings"
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
const providerBackupColumn1780000006 = "provider_backup_1780000006"
const userPluginIndex1780000006 = "CREATE INDEX `idx_trail_external_reference_user_plugin_id` ON `trail_external_reference` (`user`, `plugin_id`)"
func init() {
m.Register(func(app core.App) error {
if err := backupProviderColumn1780000006(app); err != nil {
return err
}
collection, err := app.FindCollectionByNameOrId("trail_external_reference")
if err != nil {
return err
}
// Drop+re-add (with a new id) blanks the provider column, so the
// provider-scoped unique indexes must not be rebuilt until the values
// have been restored, otherwise a cross-provider external_id clash would
// fail index creation and abort the migration.
removedIndexes := stripProviderIndexes1780000006(collection)
collection.Fields.RemoveByName("provider")
if err := collection.Fields.AddMarshaledJSONAt(3, []byte(`{
"autogeneratePattern": "",
"hidden": false,
"id": "text420001002",
"max": 128,
"min": 1,
"name": "provider",
"pattern": "^[a-z0-9][a-z0-9_-]*$",
"presentable": false,
"primaryKey": false,
"required": true,
"system": false,
"type": "text"
}`)); err != nil {
return err
}
if collection.Fields.GetByName("plugin_id") == nil {
if err := collection.Fields.AddMarshaledJSONAt(4, []byte(`{
"autogeneratePattern": "",
"hidden": false,
"id": "textpluginref",
"max": 64,
"min": 0,
"name": "plugin_id",
"pattern": "^[a-z0-9][a-z0-9_-]*$",
"presentable": false,
"primaryKey": false,
"required": false,
"system": false,
"type": "text"
}`)); err != nil {
return err
}
}
if collection.Fields.GetByName("provider_category") == nil {
if err := collection.Fields.AddMarshaledJSONAt(6, []byte(`{
"autogeneratePattern": "",
"hidden": false,
"id": "txtrmtecat01",
"max": 255,
"min": 0,
"name": "provider_category",
"pattern": "",
"presentable": false,
"primaryKey": false,
"required": false,
"system": false,
"type": "text"
}`)); err != nil {
return err
}
}
if collection.Fields.GetByName("provider_category_checked_at") == nil {
if err := collection.Fields.AddMarshaledJSONAt(7, []byte(`{
"hidden": false,
"id": "datermtecat1",
"max": "",
"min": "",
"name": "provider_category_checked_at",
"presentable": false,
"required": false,
"system": false,
"type": "date"
}`)); err != nil {
return err
}
}
if err := app.Save(collection); err != nil {
return err
}
if err := restoreProviderColumn1780000006(app); err != nil {
return err
}
collection.Indexes = append(collection.Indexes, removedIndexes...)
if !hasIndex1780000006(collection, userPluginIndex1780000006) {
collection.Indexes = append(collection.Indexes, userPluginIndex1780000006)
}
if err := app.Save(collection); err != nil {
return err
}
refs, err := app.FindAllRecords("trail_external_reference")
if err != nil {
return err
}
for _, ref := range refs {
if ref.GetString("plugin_id") != "" {
continue
}
ref.Set("plugin_id", ref.GetString("provider"))
if err := app.Save(ref); err != nil {
return err
}
}
return nil
}, func(app core.App) error {
if err := backupProviderColumn1780000006(app); err != nil {
return err
}
collection, err := app.FindCollectionByNameOrId("trail_external_reference")
if err != nil {
return err
}
removedIndexes := stripProviderIndexes1780000006(collection)
removeIndex1780000006(collection, userPluginIndex1780000006)
collection.Fields.RemoveByName("provider_category_checked_at")
collection.Fields.RemoveByName("provider_category")
collection.Fields.RemoveByName("plugin_id")
collection.Fields.RemoveByName("provider")
if err := collection.Fields.AddMarshaledJSONAt(3, []byte(`{
"hidden": false,
"id": "select420001002",
"maxSelect": 1,
"name": "provider",
"presentable": false,
"required": true,
"system": false,
"type": "select",
"values": [
"strava",
"komoot",
"hammerhead"
]
}`)); err != nil {
return err
}
if err := app.Save(collection); err != nil {
return err
}
if err := restoreProviderColumn1780000006(app); err != nil {
return err
}
collection.Indexes = append(collection.Indexes, removedIndexes...)
return app.Save(collection)
})
}
// stripProviderIndexes1780000006 removes the provider-scoped indexes from the
// collection metadata and returns them so they can be re-added once the
// provider values have been restored. PocketBase rebuilds indexes from the
// collection metadata on every save; leaving the provider indexes in place
// while the column is transiently empty risks a unique-constraint failure.
func stripProviderIndexes1780000006(collection *core.Collection) []string {
kept := make([]string, 0, len(collection.Indexes))
removed := make([]string, 0)
for _, idx := range collection.Indexes {
if strings.Contains(idx, "`provider`") {
removed = append(removed, idx)
continue
}
kept = append(kept, idx)
}
collection.Indexes = kept
return removed
}
func hasIndex1780000006(collection *core.Collection, index string) bool {
for _, existing := range collection.Indexes {
if existing == index {
return true
}
}
return false
}
func removeIndex1780000006(collection *core.Collection, index string) {
indexes := collection.Indexes[:0]
for _, existing := range collection.Indexes {
if existing == index {
continue
}
indexes = append(indexes, existing)
}
collection.Indexes = indexes
}
func backupProviderColumn1780000006(app core.App) error {
exists, err := columnExists1780000006(app, providerBackupColumn1780000006)
if err != nil {
return err
}
if exists {
return nil
}
if _, err := app.DB().
NewQuery("ALTER TABLE trail_external_reference ADD COLUMN " + providerBackupColumn1780000006 + " TEXT DEFAULT '' NOT NULL").
Execute(); err != nil {
return err
}
_, err = app.DB().
NewQuery("UPDATE trail_external_reference SET " + providerBackupColumn1780000006 + " = provider").
Execute()
return err
}
func restoreProviderColumn1780000006(app core.App) error {
if _, err := app.DB().
NewQuery("UPDATE trail_external_reference SET provider = " + providerBackupColumn1780000006).
Execute(); err != nil {
return err
}
_, err := app.DB().DropColumn("trail_external_reference", providerBackupColumn1780000006).Execute()
return err
}
func columnExists1780000006(app core.App, column string) (bool, error) {
columns, err := app.TableColumns("trail_external_reference")
if err != nil {
return false, err
}
for _, existing := range columns {
if existing == column {
return true, nil
}
}
return false, nil
}

126
db/migrations/1780566579_add_iri_to_local_resources.go Normal file
View File

@@ -0,0 +1,126 @@
package migrations
import (
"fmt"
"os"
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
func init() {
m.Register(func(app core.App) error {
origin := os.Getenv("ORIGIN")
if origin == "" {
return fmt.Errorf("ORIGIN not set")
}
// Comments
comments, err := app.FindAllRecords("comments")
if err != nil {
return err
}
for _, c := range comments {
iri := c.GetString("iri")
if iri != "" {
continue
}
iri = fmt.Sprintf("%s/api/v1/comment/%s", origin, c.Id)
c.Set("iri", iri)
if err := app.Save(c); err != nil {
return err
}
}
// ---
// Lists
lists, err := app.FindAllRecords("lists")
if err != nil {
return err
}
for _, l := range lists {
iri := l.GetString("iri")
if iri != "" {
continue
}
iri = fmt.Sprintf("%s/api/v1/list/%s", origin, l.Id)
l.Set("iri", iri)
if err := app.UnsafeWithoutHooks().Save(l); err != nil {
return err
}
}
// ---
// Summit Logs
summitLogs, err := app.FindAllRecords("summit_logs")
if err != nil {
return err
}
for _, sl := range summitLogs {
iri := sl.GetString("iri")
if iri != "" {
continue
}
iri = fmt.Sprintf("%s/api/v1/summit-log/%s", origin, sl.Id)
sl.Set("iri", iri)
if err := app.Save(sl); err != nil {
return err
}
}
// ---
// Trails
trails, err := app.FindAllRecords("trails")
if err != nil {
return err
}
for _, t := range trails {
iri := t.GetString("iri")
if iri != "" {
continue
}
iri = fmt.Sprintf("%s/api/v1/trail/%s", origin, t.Id)
t.Set("iri", iri)
if err := app.UnsafeWithoutHooks().Save(t); err != nil {
return err
}
}
// ---
// Waypoints
wps, err := app.FindAllRecords("waypoints")
if err != nil {
return err
}
for _, wp := range wps {
iri := wp.GetString("iri")
if iri != "" {
continue
}
iri = fmt.Sprintf("%s/api/v1/waypoint/%s", origin, wp.Id)
wp.Set("iri", iri)
if err := app.Save(wp); err != nil {
return err
}
}
return nil
}, func(app core.App) error {
return nil
})
}

52
db/migrations/1780734977_updated_activitypub_actors.go Normal file
View File

@@ -0,0 +1,52 @@
package migrations
import (
"github.com/pocketbase/pocketbase/core"
m "github.com/pocketbase/pocketbase/migrations"
)
func init() {
m.Register(func(app core.App) error {
collection, err := app.FindCollectionByNameOrId("pbc_1295301207")
if err != nil {
return err
}
// update field
if err := collection.Fields.AddMarshaledJSONAt(14, []byte(`{
"help": "",
"hidden": false,
"id": "bool2193750486",
"name": "is_local",
"presentable": false,
"required": false,
"system": false,
"type": "bool"
}`)); err != nil {
return err
}
return app.Save(collection)
}, func(app core.App) error {
collection, err := app.FindCollectionByNameOrId("pbc_1295301207")
if err != nil {
return err
}
// update field
if err := collection.Fields.AddMarshaledJSONAt(14, []byte(`{
"help": "",
"hidden": false,
"id": "bool2193750486",
"name": "isLocal",
"presentable": false,
"required": false,
"system": false,
"type": "bool"
}`)); err != nil {
return err
}
return app.Save(collection)
})
}

BIN
db/migrations/initial_data/other.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 432 KiB

912
db/plugins/importer/importer.go Normal file
View File

@@ -0,0 +1,912 @@
package importer
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"math"
"mime"
"net/http"
"net/url"
urlpath "path"
"path/filepath"
"slices"
"strings"
"time"
"pocketbase/pluginsystem"
"pocketbase/util"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/filesystem"
"github.com/tkrajina/gpxgo/gpx"
)
type Options struct {
UserID string
ActorID string
DefaultPublic bool
CreateSummitLogForCompleted bool
CategoryMapping map[string]string
Manifest pluginsystem.Manifest
Policy pluginsystem.RequestPolicyContext
Auth map[string]any
}
// Result tells the sync loop whether a plugin item created a new trail or was
// skipped because the same provider/external id had already been imported.
type Result struct {
TrailID string
Created bool
Skipped bool
}
// ImportTrail is the boundary between plugin output and wanderer records. It
// validates the provider identity, deduplicates by trail_external_reference,
// stores the GPX/photos, maps GPX metrics onto the trail record, and creates the
// optional related waypoints and summit log.
func ImportTrail(ctx context.Context, app core.App, item pluginsystem.TrailImport, opts Options) (*Result, error) {
if item.Source.Provider == "" || item.Source.ExternalID == "" {
return nil, fmt.Errorf("source provider and externalId are required")
}
if existing, err := util.FindTrailByExternalReferenceForUser(app, opts.UserID, item.Source.Provider, item.Source.ExternalID); err != nil {
return nil, err
} else if existing != nil {
return &Result{TrailID: existing.Id, Skipped: true}, nil
}
gpxBytes, parsedGPX, err := decodeAndParseGPX(item.Track)
if err != nil {
return nil, err
}
gpxFile, err := filesystem.NewFileFromBytes(gpxBytes, safeGPXFileName(item.Name))
if err != nil {
return nil, err
}
collection, err := app.FindCollectionByNameOrId("trails")
if err != nil {
return nil, err
}
record := core.NewRecord(collection)
metrics := metricsFromGPX(parsedGPX)
trackIndex := trackDistanceIndexFromGPX(parsedGPX)
applyProviderStart(&metrics, trackIndex, item.Metadata)
applyProviderMetrics(&metrics, item.Metadata)
public := publicFromPrivacy(item.Privacy, opts.DefaultPublic)
categoryID := categoryIDForImport(app, item, opts.CategoryMapping)
date := dateFromImport(item, metrics)
mediaBudget := &pluginMediaBudget{}
photos := photoFiles(ctx, app, item.Photos, opts, mediaBudget)
record.Load(map[string]any{
"name": fallbackName(item.Name),
"description": item.Description,
"public": public,
"completed": item.Kind == "completed",
"distance": metrics.Distance,
"elevation_gain": metrics.ElevationGain,
"elevation_loss": metrics.ElevationLoss,
"duration": metrics.Duration,
"date": date,
"lat": metrics.StartLat,
"lon": metrics.StartLon,
"difficulty": "easy",
"category": categoryID,
"author": opts.ActorID,
})
record.Set("gpx", gpxFile)
if len(photos) > 0 {
record.Set("photos", photos)
}
if err := app.Save(record); err != nil {
return nil, err
}
if err := util.EnsureTrailExternalReference(app, record.Id, item.Source.Provider, item.Source.ExternalID, opts.Manifest.ID, ProviderCategoryFromImport(item)); err != nil {
return nil, err
}
if err := createWaypoints(ctx, app, item.Waypoints, opts, mediaBudget, record.Id, trackIndex); err != nil {
return nil, err
}
if opts.CreateSummitLogForCompleted && item.Kind == "completed" {
if err := createSummitLog(app, record.Id, opts.ActorID, date, metrics); err != nil {
return nil, err
}
}
return &Result{TrailID: record.Id, Created: true}, nil
}
type trailMetrics struct {
Distance float64
ElevationGain float64
ElevationLoss float64
Duration float64
StartLat float64
StartLon float64
StartTime time.Time
}
type geoPoint struct {
Lat float64
Lon float64
}
type trackDistanceIndex struct {
points []indexedTrackPoint
segments []indexedTrackSegment
}
type indexedTrackPoint struct {
point geoPoint
distance float64
}
type indexedTrackSegment struct {
start geoPoint
end geoPoint
startDistance float64
length float64
}
const maxProviderStartDistanceMeters = 1000
// decodeAndParseGPX keeps the importer strict for now: plugins must return GPX
// as base64 so the host can compute canonical trail metrics itself.
func decodeAndParseGPX(track pluginsystem.Track) ([]byte, *gpx.GPX, error) {
if track.Format != "gpx" {
return nil, nil, fmt.Errorf("unsupported track format %q", track.Format)
}
if track.ContentBase64 == "" {
return nil, nil, fmt.Errorf("track contentBase64 is required")
}
content, err := base64.StdEncoding.DecodeString(track.ContentBase64)
if err != nil {
return nil, nil, fmt.Errorf("decode GPX: %w", err)
}
parsed, err := gpx.Parse(bytes.NewReader(content))
if err != nil {
return nil, nil, fmt.Errorf("parse GPX: %w", err)
}
return content, parsed, nil
}
// metricsFromGPX derives fallback trail fields from the GPX. Provider metadata
// may override summary metrics and, when plausible, the displayed start point.
func metricsFromGPX(gpxData *gpx.GPX) trailMetrics {
uphillDownhill := gpxData.UphillDownhill()
movingData := gpxData.MovingData()
timeBounds := gpxData.TimeBounds()
metrics := trailMetrics{
Distance: gpxData.Length2D(),
ElevationGain: uphillDownhill.Uphill,
ElevationLoss: uphillDownhill.Downhill,
Duration: movingData.MovingTime + movingData.StoppedTime,
StartTime: timeBounds.StartTime,
}
for _, track := range gpxData.Tracks {
for _, segment := range track.Segments {
if len(segment.Points) == 0 {
continue
}
metrics.StartLat = segment.Points[0].Latitude
metrics.StartLon = segment.Points[0].Longitude
return metrics
}
}
return metrics
}
// applyProviderStart lets providers correct the displayed trail start when the
// provider's intended start is close to the imported GPX track. Implausible
// starts are ignored so broken metadata does not move trails off their geometry.
func applyProviderStart(metrics *trailMetrics, trackIndex trackDistanceIndex, metadata map[string]any) {
if metrics == nil || len(metadata) == 0 {
return
}
start, ok := providerStartFromMetadata(metadata)
if !ok || !providerStartNearTrack(trackIndex, start) {
return
}
metrics.StartLat = start.Lat
metrics.StartLon = start.Lon
}
func providerStartFromMetadata(metadata map[string]any) (geoPoint, bool) {
raw, ok := metadata["providerStart"]
if !ok {
return geoPoint{}, false
}
values, ok := raw.(map[string]any)
if !ok {
return geoPoint{}, false
}
lat, ok := floatMetadata(values, "lat")
if !ok {
lat, ok = floatMetadata(values, "latitude")
}
if !ok {
return geoPoint{}, false
}
lon, ok := floatMetadata(values, "lon")
if !ok {
lon, ok = floatMetadata(values, "longitude")
}
if !ok || lat < -90 || lat > 90 || lon < -180 || lon > 180 {
return geoPoint{}, false
}
return geoPoint{Lat: lat, Lon: lon}, true
}
func providerStartNearTrack(trackIndex trackDistanceIndex, start geoPoint) bool {
distance, ok := trackIndex.nearest(start)
return ok && distance.offTrack <= maxProviderStartDistanceMeters
}
type trackDistance struct {
fromStart float64
offTrack float64
}
func trackDistanceIndexFromGPX(gpxData *gpx.GPX) trackDistanceIndex {
index := trackDistanceIndex{}
if gpxData == nil {
return index
}
totalDistance := 0.0
for _, track := range gpxData.Tracks {
for _, segment := range track.Segments {
var previous geoPoint
hasPrevious := false
for _, point := range segment.Points {
current := geoPoint{Lat: point.Latitude, Lon: point.Longitude}
if !hasPrevious {
index.points = append(index.points, indexedTrackPoint{
point: current,
distance: totalDistance,
})
previous = current
hasPrevious = true
continue
}
length := util.HaversineDistanceMeters(previous.Lat, previous.Lon, current.Lat, current.Lon)
if length > 0 {
index.segments = append(index.segments, indexedTrackSegment{
start: previous,
end: current,
startDistance: totalDistance,
length: length,
})
totalDistance += length
}
index.points = append(index.points, indexedTrackPoint{
point: current,
distance: totalDistance,
})
previous = current
}
}
}
return index
}
func (index trackDistanceIndex) nearest(point geoPoint) (trackDistance, bool) {
var nearest trackDistance
found := false
for _, candidate := range index.points {
offTrack := util.HaversineDistanceMeters(point.Lat, point.Lon, candidate.point.Lat, candidate.point.Lon)
if !found || offTrack < nearest.offTrack {
nearest = trackDistance{fromStart: candidate.distance, offTrack: offTrack}
found = true
}
}
for _, segment := range index.segments {
offTrack, t := pointToSegmentProjectionMeters(point, segment.start, segment.end)
fromStart := segment.startDistance + segment.length*t
if !found || offTrack < nearest.offTrack {
nearest = trackDistance{fromStart: fromStart, offTrack: offTrack}
found = true
}
}
return nearest, found
}
func pointToSegmentProjectionMeters(point geoPoint, start geoPoint, end geoPoint) (float64, float64) {
const earthRadius = 6371000.0
latRad := point.Lat * math.Pi / 180
toXY := func(p geoPoint) (float64, float64) {
x := (p.Lon - point.Lon) * math.Pi / 180 * math.Cos(latRad) * earthRadius
y := (p.Lat - point.Lat) * math.Pi / 180 * earthRadius
return x, y
}
startX, startY := toXY(start)
endX, endY := toXY(end)
dx := endX - startX
dy := endY - startY
lengthSquared := dx*dx + dy*dy
if lengthSquared == 0 {
return math.Hypot(startX, startY), 0
}
t := -(startX*dx + startY*dy) / lengthSquared
if t < 0 {
t = 0
} else if t > 1 {
t = 1
}
closestX := startX + t*dx
closestY := startY + t*dy
return math.Hypot(closestX, closestY), t
}
// applyProviderMetrics lets plugins preserve provider-provided summary metrics
// where those values are more authoritative than values recalculated from a
// simplified/import GPX. GPX parsing remains mandatory and provides fallback
// metrics plus the start coordinate.
func applyProviderMetrics(metrics *trailMetrics, metadata map[string]any) {
if metrics == nil || len(metadata) == 0 {
return
}
if value, ok := positiveFloatMetadata(metadata, "distance"); ok {
metrics.Distance = value
}
if value, ok := positiveFloatMetadata(metadata, "elevationGain"); ok {
metrics.ElevationGain = value
}
if value, ok := positiveFloatMetadata(metadata, "elevationLoss"); ok {
metrics.ElevationLoss = value
}
if value, ok := positiveFloatMetadata(metadata, "duration"); ok {
metrics.Duration = value
}
}
func positiveFloatMetadata(metadata map[string]any, key string) (float64, bool) {
value, ok := floatMetadata(metadata, key)
return value, ok && value > 0
}
func floatMetadata(metadata map[string]any, key string) (float64, bool) {
switch value := metadata[key].(type) {
case float64:
return value, true
case float32:
floatValue := float64(value)
return floatValue, true
case int:
floatValue := float64(value)
return floatValue, true
case int64:
floatValue := float64(value)
return floatValue, true
case int32:
floatValue := float64(value)
return floatValue, true
case json.Number:
parsed, err := value.Float64()
return parsed, err == nil
default:
return 0, false
}
}
// publicFromPrivacy respects explicit provider privacy when present and falls
// back to the user's wanderer default when the plugin leaves privacy unset.
func publicFromPrivacy(privacy *string, defaultPublic bool) bool {
if privacy == nil || *privacy == "" {
return defaultPublic
}
return *privacy == "public"
}
// dateFromImport chooses the best available trail date: provider start time,
// GPX start time, then the import time.
func dateFromImport(item pluginsystem.TrailImport, metrics trailMetrics) time.Time {
if item.StartedAt != nil {
return *item.StartedAt
}
if !metrics.StartTime.IsZero() {
return metrics.StartTime
}
return time.Now()
}
// createWaypoints persists plugin-provided waypoints after the trail exists so
// they can reference the imported trail record.
func createWaypoints(ctx context.Context, app core.App, waypoints []pluginsystem.Waypoint, opts Options, mediaBudget *pluginMediaBudget, trailID string, trackIndex trackDistanceIndex) error {
if len(waypoints) == 0 {
return nil
}
if err := ctx.Err(); err != nil {
return err
}
collection, err := app.FindCollectionByNameOrId("waypoints")
if err != nil {
return err
}
for _, waypoint := range waypoints {
record := core.NewRecord(collection)
icon := waypoint.Icon
if icon == "" {
icon = "circle"
}
distanceFromStart := 0.0
if distance, ok := trackIndex.nearest(geoPoint{Lat: waypoint.Lat, Lon: waypoint.Lon}); ok {
distanceFromStart = distance.fromStart
}
photos := photoFiles(ctx, app, waypoint.Photos, opts, mediaBudget)
record.Load(map[string]any{
"name": waypoint.Name,
"description": waypoint.Description,
"lat": waypoint.Lat,
"lon": waypoint.Lon,
"icon": icon,
"author": opts.ActorID,
"distance_from_start": distanceFromStart,
"trail": trailID,
})
if len(photos) > 0 {
record.Set("photos", photos)
}
if err := app.Save(record); err != nil {
return err
}
}
return nil
}
// photoFiles converts plugin photo descriptors into PocketBase file objects.
// Individual photo failures are logged and skipped so one broken media URL does
// not fail the whole trail import.
type pluginMediaBudget struct {
items int
bytes int64
}
func (b *pluginMediaBudget) remainingBytes() int64 {
remaining := util.DefaultPluginMaxImportMediaBytes - b.bytes
if remaining < util.DefaultPluginMediaMaxBytes {
return remaining
}
return util.DefaultPluginMediaMaxBytes
}
func photoFiles(ctx context.Context, app core.App, photos []pluginsystem.Photo, opts Options, budget *pluginMediaBudget) []*filesystem.File {
if len(photos) == 0 {
return nil
}
files := make([]*filesystem.File, 0, len(photos))
now := time.Now()
for _, photo := range photos {
if budget.items >= util.DefaultPluginMaxImportMediaItems {
app.Logger().Warn("skipping plugin photo because media item limit was reached", "limit", util.DefaultPluginMaxImportMediaItems)
continue
}
if err := ctx.Err(); err != nil {
app.Logger().Warn("skipping plugin photo because import context was cancelled", "error", err)
return files
}
if photo.Source.ExpiresAt != nil && photo.Source.ExpiresAt.Before(now) {
app.Logger().Warn("skipping expired plugin photo", "external_id", photo.ExternalID)
continue
}
maxBytes := budget.remainingBytes()
if maxBytes <= 0 {
app.Logger().Warn("skipping plugin photo because aggregate media byte limit was reached", "external_id", photo.ExternalID, "limit", util.DefaultPluginMaxImportMediaBytes)
continue
}
file, bytesRead, err := photoFile(ctx, photo, opts, maxBytes)
if err != nil {
app.Logger().Warn("skipping plugin photo", "external_id", photo.ExternalID, "error", err)
continue
}
if file != nil {
files = append(files, file)
budget.items++
budget.bytes += bytesRead
}
}
return files
}
// photoFile fetches one plugin-provided photo source. URL sources are validated
// before PocketBase performs the server-side download.
func photoFile(ctx context.Context, photo pluginsystem.Photo, opts Options, maxBytes int64) (*filesystem.File, int64, error) {
switch photo.Source.Type {
case "url":
if photo.Source.URL == "" {
return nil, 0, fmt.Errorf("photo URL is empty")
}
if err := validateRemoteMediaURLSyntax(photo.Source.URL); err != nil {
return nil, 0, err
}
fetched, err := util.FetchPublicURL(ctx, photo.Source.URL, maxBytes)
if err != nil {
return nil, 0, err
}
file, err := filesystem.NewFileFromBytes(fetched.Body, safeMediaFileName(photo.Filename, urlPathBase(fetched.FinalURL), fetched.ContentType, photo.ContentType))
return file, int64(len(fetched.Body)), err
case "connector":
fetched, err := fetchConnectorMedia(ctx, photo, opts, maxBytes)
if err != nil {
return nil, 0, err
}
file, err := filesystem.NewFileFromBytes(fetched.Body, safeMediaFileName(photo.Filename, urlPathBase(fetched.FinalURL), fetched.ContentType, photo.ContentType))
return file, int64(len(fetched.Body)), err
default:
return nil, 0, fmt.Errorf("unsupported photo source type %q", photo.Source.Type)
}
}
func fetchConnectorMedia(ctx context.Context, photo pluginsystem.Photo, opts Options, maxBytes int64) (*util.SafeFetchResult, error) {
if photo.Source.MediaRef == nil {
return nil, fmt.Errorf("connector mediaRef is required")
}
ref := *photo.Source.MediaRef
if ref.AssetID != "" && ref.Path == "" {
return nil, fmt.Errorf("mediaRef.assetId is metadata only; path is required")
}
target := pluginsystem.RequestTarget{
Type: "connector",
Connector: ref.Connector,
Path: ref.Path,
Query: ref.Query,
}
resolved, err := pluginsystem.ResolveRequestTarget(opts.Manifest, target, opts.Policy)
if err != nil {
return nil, err
}
if ref.Auth != "" {
if !resolved.Connector.SupportsMediaAuth {
return nil, fmt.Errorf("connector %q does not support media auth", ref.Connector)
}
if len(resolved.Connector.Auth) > 0 && !slices.Contains(resolved.Connector.Auth, ref.Auth) {
return nil, fmt.Errorf("auth context %q is not permitted for connector %q", ref.Auth, ref.Connector)
}
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, resolved.URL.String(), nil)
if err != nil {
return nil, err
}
if err := pluginsystem.InjectRequestAuthForContext(opts.Manifest, opts.Auth, ref.Auth, req); err != nil {
return nil, err
}
var storageRedirect *storageRedirectTarget
client, err := util.ConnectorHTTPClient(util.ConnectorHTTPPolicy{
BaseURL: resolved.Connector.BaseURL,
AllowPrivate: resolved.Connector.AllowPrivate,
TLSMode: resolved.Connector.TLS.Mode,
TLSCABundle: resolved.Connector.TLS.CABundle,
}, func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return fmt.Errorf("too many redirects")
}
previous := resolved.URL
if len(via) > 0 {
previous = via[len(via)-1].URL
}
if err := pluginsystem.ValidateConnectorRedirect(resolved.Connector, previous, req.URL); err == nil {
return nil
}
origin, err := pluginsystem.ConnectorStorageRedirectOrigin(resolved.Connector, previous, req.URL)
if err != nil {
return err
}
stripConnectorAuth(req, opts.Manifest, ref.Auth)
storageRedirect = &storageRedirectTarget{
URL: req.URL.String(),
Origin: origin,
}
return http.ErrUseLastResponse
})
if err != nil {
return nil, err
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if storageRedirect != nil && resp.StatusCode >= 300 && resp.StatusCode < 400 {
return fetchStorageRedirectMedia(ctx, *storageRedirect, maxBytes)
}
body, err := util.ReadBoundedForPlugin(resp.Body, maxBytes)
if err != nil {
return nil, err
}
return &util.SafeFetchResult{Body: body, ContentType: resp.Header.Get("Content-Type"), FinalURL: resp.Request.URL.String()}, nil
}
type storageRedirectTarget struct {
URL string
Origin pluginsystem.ResolvedConnectorOrigin
}
func fetchStorageRedirectMedia(ctx context.Context, redirect storageRedirectTarget, maxBytes int64) (*util.SafeFetchResult, error) {
storageConnector := pluginsystem.ResolvedConnectorTarget{
Name: redirect.Origin.Name,
BaseURL: redirect.Origin.BaseURL,
BasePath: redirect.Origin.BasePath,
AllowPrivate: redirect.Origin.AllowPrivate,
TLS: redirect.Origin.TLS,
AllowedPathPrefixes: []string{"/"},
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, redirect.URL, nil)
if err != nil {
return nil, err
}
client, err := util.ConnectorHTTPClient(util.ConnectorHTTPPolicy{
BaseURL: redirect.Origin.BaseURL,
AllowPrivate: redirect.Origin.AllowPrivate,
TLSMode: redirect.Origin.TLS.Mode,
TLSCABundle: redirect.Origin.TLS.CABundle,
}, func(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return fmt.Errorf("too many redirects")
}
previous := req.URL
if len(via) > 0 {
previous = via[len(via)-1].URL
}
return pluginsystem.ValidateConnectorRedirect(storageConnector, previous, req.URL)
})
if err != nil {
return nil, err
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := util.ReadBoundedForPlugin(resp.Body, maxBytes)
if err != nil {
return nil, err
}
return &util.SafeFetchResult{Body: body, ContentType: resp.Header.Get("Content-Type"), FinalURL: resp.Request.URL.String()}, nil
}
func stripConnectorAuth(req *http.Request, manifest pluginsystem.Manifest, authName string) {
req.Header.Del(pluginsystem.AuthHeaderAuthorization)
if authName == "" {
return
}
authContext, ok := manifest.Auth.Contexts[authName]
if !ok {
return
}
if authContext.Name != "" {
req.Header.Del(authContext.Name)
req.URL.RawQuery = removeRawQueryParamOrdered(req.URL.RawQuery, authContext.Name)
}
if authContext.SecretField != "" {
req.Header.Del(authContext.SecretField)
req.URL.RawQuery = removeRawQueryParamOrdered(req.URL.RawQuery, authContext.SecretField)
}
}
func validateRemoteMediaURLSyntax(rawURL string) error {
parsed, err := url.Parse(rawURL)
if err != nil {
return fmt.Errorf("invalid media URL: %w", err)
}
if parsed.Scheme != "http" && parsed.Scheme != "https" {
return fmt.Errorf("unsupported media URL scheme %q", parsed.Scheme)
}
host := parsed.Hostname()
if host == "" {
return fmt.Errorf("media URL has no host")
}
return nil
}
func urlPathBase(rawURL string) string {
parsed, err := url.Parse(rawURL)
if err != nil {
return ""
}
return urlpath.Base(parsed.Path)
}
func removeRawQueryParamOrdered(rawQuery string, name string) string {
if rawQuery == "" || name == "" {
return rawQuery
}
parts := strings.Split(rawQuery, "&")
kept := make([]string, 0, len(parts))
for _, part := range parts {
if part == "" {
continue
}
rawName := part
if idx := strings.Index(rawName, "="); idx >= 0 {
rawName = rawName[:idx]
}
decodedName, err := url.QueryUnescape(rawName)
if err == nil && decodedName == name {
continue
}
kept = append(kept, part)
}
return strings.Join(kept, "&")
}
// createSummitLog mirrors completed imported trails into summit_logs when the
// user has enabled that compatibility option.
func createSummitLog(app core.App, trailID string, actorID string, date time.Time, metrics trailMetrics) error {
collection, err := app.FindCollectionByNameOrId("summit_logs")
if err != nil {
return err
}
record := core.NewRecord(collection)
record.Load(map[string]any{
"distance": metrics.Distance,
"elevation_gain": metrics.ElevationGain,
"elevation_loss": metrics.ElevationLoss,
"duration": metrics.Duration,
"date": date,
"author": actorID,
"trail": trailID,
})
return app.Save(record)
}
func categoryIDForImport(app core.App, item pluginsystem.TrailImport, mapping map[string]string) string {
if category, matched := CategoryFromProviderMapping(app, ProviderCategoryFromImport(item), mapping); matched {
return category
}
return categoryIDForActivityType(app, item.ActivityType)
}
func ProviderCategoryFromImport(item pluginsystem.TrailImport) string {
value, _ := item.Metadata["providerCategory"].(string)
if strings.TrimSpace(value) != "" {
return strings.TrimSpace(value)
}
value, _ = item.Metadata["sourceSport"].(string)
return strings.TrimSpace(value)
}
func CategoryFromProviderMapping(app core.App, providerCategory string, mapping map[string]string) (string, bool) {
providerCategory = strings.TrimSpace(providerCategory)
if providerCategory == "" || len(mapping) == 0 {
return "", false
}
rawTarget, matched := mapping[providerCategory]
if !matched {
return "", false
}
target := strings.TrimSpace(rawTarget)
if target == "" {
return "", true
}
if category, err := app.FindRecordById("categories", target); err == nil && category != nil {
return category.Id, true
}
category, _ := app.FindFirstRecordByData("categories", "name", target)
if category == nil {
return "", false
}
return category.Id, true
}
// categoryIDForActivityType maps common provider activity labels to wanderer's
// built-in categories. Unknown labels intentionally leave the category empty.
func categoryIDForActivityType(app core.App, activityType string) string {
categoryMap := map[string]string{
"hiking": "Hiking",
"hike": "Hiking",
"walking": "Walking",
"walk": "Walking",
"running": "Walking",
"run": "Walking",
"biking": "Biking",
"cycling": "Biking",
"ride": "Biking",
"mtb": "Biking",
"skiing": "Skiing",
"canoeing": "Canoeing",
"climbing": "Climbing",
}
name := categoryMap[strings.ToLower(activityType)]
if name == "" {
return ""
}
category, _ := app.FindFirstRecordByData("categories", "name", name)
if category == nil {
return ""
}
return category.Id
}
func fallbackName(name string) string {
if strings.TrimSpace(name) != "" {
return name
}
return "Imported trail"
}
// safeGPXFileName turns provider trail names into filesystem-safe GPX filenames.
func safeGPXFileName(name string) string {
name = strings.TrimSpace(name)
if name == "" {
name = "imported-trail"
}
name = filepath.Base(name)
name = strings.TrimSuffix(name, filepath.Ext(name))
name = strings.Map(func(r rune) rune {
switch r {
case '/', '\\', ':', '*', '?', '"', '<', '>', '|':
return '-'
default:
return r
}
}, name)
return name + ".gpx"
}
// safeMediaFileName picks the first safe candidate filename and adds a best
// effort extension when providers only expose a content type.
func safeMediaFileName(candidates ...string) string {
filename := ""
for _, candidate := range candidates {
candidate = strings.TrimSpace(candidate)
if candidate == "" || strings.Contains(candidate, "/") {
continue
}
base := filepath.Base(candidate)
if base == "." || base == ".." {
continue
}
filename = candidate
break
}
if filename == "" {
filename = "photo"
}
filename = filepath.Base(filename)
filename = strings.Map(func(r rune) rune {
switch r {
case '/', '\\', ':', '*', '?', '"', '<', '>', '|':
return '-'
default:
return r
}
}, filename)
if ext := filepath.Ext(filename); ext == "" || ext == "." {
filename += extensionFromContentTypes(candidates...)
}
return filename
}
func extensionFromContentTypes(candidates ...string) string {
for _, candidate := range candidates {
if extensions, err := mime.ExtensionsByType(strings.TrimSpace(candidate)); err == nil && len(extensions) > 0 {
return extensions[0]
}
}
return ".jpg"
}

432
db/plugins/importer/importer_test.go Normal file
View File

@@ -0,0 +1,432 @@
package importer
import (
"context"
"encoding/base64"
"strings"
"testing"
"time"
pluginsystem "pocketbase/pluginsystem"
"pocketbase/util"
)
const sampleGPX = `<?xml version="1.0" encoding="UTF-8"?>
<gpx version="1.1" creator="test">
<trk><trkseg>
<trkpt lat="46.000000" lon="8.000000"><ele>100</ele><time>2026-01-01T10:00:00Z</time></trkpt>
<trkpt lat="46.001000" lon="8.001000"><ele>120</ele><time>2026-01-01T10:10:00Z</time></trkpt>
</trkseg></trk>
</gpx>`
func gpxTrack() pluginsystem.Track {
return pluginsystem.Track{
Format: "gpx",
ContentBase64: base64.StdEncoding.EncodeToString([]byte(sampleGPX)),
}
}
func TestDecodeAndParseGPX(t *testing.T) {
t.Run("valid", func(t *testing.T) {
raw, parsed, err := decodeAndParseGPX(gpxTrack())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if parsed == nil {
t.Fatal("expected parsed gpx")
}
if string(raw) != sampleGPX {
t.Fatal("decoded bytes do not match input")
}
})
t.Run("unsupported format", func(t *testing.T) {
if _, _, err := decodeAndParseGPX(pluginsystem.Track{Format: "tcx", ContentBase64: "x"}); err == nil {
t.Fatal("expected error for unsupported format")
}
})
t.Run("empty content", func(t *testing.T) {
if _, _, err := decodeAndParseGPX(pluginsystem.Track{Format: "gpx"}); err == nil {
t.Fatal("expected error for empty content")
}
})
t.Run("invalid base64", func(t *testing.T) {
if _, _, err := decodeAndParseGPX(pluginsystem.Track{Format: "gpx", ContentBase64: "!!!not-base64"}); err == nil {
t.Fatal("expected error for invalid base64")
}
})
t.Run("invalid gpx", func(t *testing.T) {
track := pluginsystem.Track{Format: "gpx", ContentBase64: base64.StdEncoding.EncodeToString([]byte("not gpx"))}
if _, _, err := decodeAndParseGPX(track); err == nil {
t.Fatal("expected error for invalid gpx")
}
})
}
func TestMetricsFromGPX(t *testing.T) {
_, parsed, err := decodeAndParseGPX(gpxTrack())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
metrics := metricsFromGPX(parsed)
if metrics.StartLat != 46.0 || metrics.StartLon != 8.0 {
t.Fatalf("unexpected start point: %v, %v", metrics.StartLat, metrics.StartLon)
}
if metrics.Distance <= 0 {
t.Fatalf("expected positive distance, got %v", metrics.Distance)
}
if metrics.ElevationGain <= 0 {
t.Fatalf("expected positive elevation gain, got %v", metrics.ElevationGain)
}
if !metrics.StartTime.Equal(time.Date(2026, 1, 1, 10, 0, 0, 0, time.UTC)) {
t.Fatalf("unexpected start time: %v", metrics.StartTime)
}
}
func TestApplyProviderMetrics(t *testing.T) {
metrics := trailMetrics{
Distance: 1,
ElevationGain: 2,
ElevationLoss: 3,
Duration: 4,
StartLat: 46,
StartLon: 8,
}
applyProviderMetrics(&metrics, map[string]any{
"distance": 1234.5,
"elevationGain": 234.5,
"elevationLoss": 45.5,
"duration": 3600,
})
if metrics.Distance != 1234.5 {
t.Fatalf("distance = %v", metrics.Distance)
}
if metrics.ElevationGain != 234.5 {
t.Fatalf("elevation gain = %v", metrics.ElevationGain)
}
if metrics.ElevationLoss != 45.5 {
t.Fatalf("elevation loss = %v", metrics.ElevationLoss)
}
if metrics.Duration != 3600 {
t.Fatalf("duration = %v", metrics.Duration)
}
if metrics.StartLat != 46 || metrics.StartLon != 8 {
t.Fatalf("provider metadata must not override start point")
}
}
func TestApplyProviderStart(t *testing.T) {
_, parsed, err := decodeAndParseGPX(gpxTrack())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
trackIndex := trackDistanceIndexFromGPX(parsed)
t.Run("uses plausible provider start", func(t *testing.T) {
metrics := metricsFromGPX(parsed)
applyProviderStart(&metrics, trackIndex, map[string]any{
"providerStart": map[string]any{
"lat": 45.9995,
"lon": 7.9995,
},
})
if metrics.StartLat != 45.9995 || metrics.StartLon != 7.9995 {
t.Fatalf("unexpected provider start: %v, %v", metrics.StartLat, metrics.StartLon)
}
})
t.Run("ignores distant provider start", func(t *testing.T) {
metrics := metricsFromGPX(parsed)
applyProviderStart(&metrics, trackIndex, map[string]any{
"providerStart": map[string]any{
"lat": 47.0,
"lon": 8.0,
},
})
if metrics.StartLat != 46.0 || metrics.StartLon != 8.0 {
t.Fatalf("distant provider start should be ignored: %v, %v", metrics.StartLat, metrics.StartLon)
}
})
t.Run("ignores invalid provider start", func(t *testing.T) {
metrics := metricsFromGPX(parsed)
applyProviderStart(&metrics, trackIndex, map[string]any{
"providerStart": map[string]any{
"lat": 91.0,
"lon": 8.0,
},
})
if metrics.StartLat != 46.0 || metrics.StartLon != 8.0 {
t.Fatalf("invalid provider start should be ignored: %v, %v", metrics.StartLat, metrics.StartLon)
}
})
}
func TestTrackDistanceIndexNearest(t *testing.T) {
_, parsed, err := decodeAndParseGPX(gpxTrack())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
trackIndex := trackDistanceIndexFromGPX(parsed)
total := util.HaversineDistanceMeters(46.0, 8.0, 46.001, 8.001)
t.Run("start point", func(t *testing.T) {
distance, ok := trackIndex.nearest(geoPoint{Lat: 46.0, Lon: 8.0})
if !ok {
t.Fatal("expected nearest distance")
}
if distance.fromStart != 0 {
t.Fatalf("got %v, want 0", distance.fromStart)
}
})
t.Run("mid segment projection", func(t *testing.T) {
distance, ok := trackIndex.nearest(geoPoint{Lat: 46.0005, Lon: 8.0005})
if !ok {
t.Fatal("expected nearest distance")
}
if distance.fromStart < total*0.45 || distance.fromStart > total*0.55 {
t.Fatalf("got %v, want about half of %v", distance.fromStart, total)
}
})
t.Run("end point", func(t *testing.T) {
distance, ok := trackIndex.nearest(geoPoint{Lat: 46.001, Lon: 8.001})
if !ok {
t.Fatal("expected nearest distance")
}
if distance.fromStart < total-0.001 || distance.fromStart > total+0.001 {
t.Fatalf("got %v, want %v", distance.fromStart, total)
}
})
}
func TestApplyProviderMetricsIgnoresEmptyValues(t *testing.T) {
metrics := trailMetrics{
Distance: 1,
ElevationGain: 2,
ElevationLoss: 3,
Duration: 4,
}
applyProviderMetrics(&metrics, map[string]any{
"distance": 0,
"elevationGain": -1,
"elevationLoss": "",
"duration": nil,
})
if metrics.Distance != 1 || metrics.ElevationGain != 2 || metrics.ElevationLoss != 3 || metrics.Duration != 4 {
t.Fatalf("unexpected metrics after empty metadata: %#v", metrics)
}
}
func TestPublicFromPrivacy(t *testing.T) {
public := "public"
private := "private"
empty := ""
cases := []struct {
name string
privacy *string
defaultPublic bool
want bool
}{
{"nil keeps default true", nil, true, true},
{"nil keeps default false", nil, false, false},
{"explicit public", &public, false, true},
{"explicit private", &private, true, false},
{"empty keeps default", &empty, true, true},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if got := publicFromPrivacy(tc.privacy, tc.defaultPublic); got != tc.want {
t.Fatalf("got %v, want %v", got, tc.want)
}
})
}
}
func TestCategoryIDForImportDoesNotFallbackWhenProviderMappingIsBlank(t *testing.T) {
item := pluginsystem.TrailImport{
ActivityType: "biking",
Metadata: map[string]any{
"providerCategory": " Ride ",
},
}
if got := categoryIDForImport(nil, item, map[string]string{"Ride": ""}); got != "" {
t.Fatalf("expected blank provider mapping to suppress activity fallback, got %q", got)
}
}
func TestProviderCategoryFromImport(t *testing.T) {
if got := ProviderCategoryFromImport(pluginsystem.TrailImport{
Metadata: map[string]any{"providerCategory": " Ride "},
}); got != "Ride" {
t.Fatalf("got %q", got)
}
if got := ProviderCategoryFromImport(pluginsystem.TrailImport{
Metadata: map[string]any{"sourceSport": " hiking "},
}); got != "hiking" {
t.Fatalf("got %q", got)
}
}
func TestDateFromImport(t *testing.T) {
started := time.Date(2025, 6, 1, 8, 0, 0, 0, time.UTC)
t.Run("uses StartedAt", func(t *testing.T) {
item := pluginsystem.TrailImport{StartedAt: &started}
if got := dateFromImport(item, trailMetrics{}); !got.Equal(started) {
t.Fatalf("got %v, want %v", got, started)
}
})
t.Run("falls back to metrics start time", func(t *testing.T) {
metricStart := time.Date(2024, 1, 2, 3, 0, 0, 0, time.UTC)
if got := dateFromImport(pluginsystem.TrailImport{}, trailMetrics{StartTime: metricStart}); !got.Equal(metricStart) {
t.Fatalf("got %v, want %v", got, metricStart)
}
})
t.Run("falls back to now", func(t *testing.T) {
got := dateFromImport(pluginsystem.TrailImport{}, trailMetrics{})
if time.Since(got) > time.Minute {
t.Fatalf("expected ~now, got %v", got)
}
})
}
func TestFallbackName(t *testing.T) {
if got := fallbackName("My Trail"); got != "My Trail" {
t.Fatalf("got %q", got)
}
if got := fallbackName(""); got != "Imported trail" {
t.Fatalf("got %q", got)
}
if got := fallbackName(" "); got != "Imported trail" {
t.Fatalf("got %q", got)
}
}
func TestSafeGPXFileName(t *testing.T) {
cases := map[string]string{
"track.gpx": "track.gpx",
"My Trip": "My Trip.gpx",
"": "imported-trail.gpx",
"../../etc/passwd": "passwd.gpx",
"a:b*c?": "a-b-c-.gpx",
}
for in, want := range cases {
if got := safeGPXFileName(in); got != want {
t.Fatalf("safeGPXFileName(%q) = %q, want %q", in, got, want)
}
}
}
func TestSafeMediaFileName(t *testing.T) {
t.Run("keeps valid filename", func(t *testing.T) {
if got := safeMediaFileName("photo.jpg"); got != "photo.jpg" {
t.Fatalf("got %q", got)
}
})
t.Run("skips empty and slashed candidates", func(t *testing.T) {
if got := safeMediaFileName("", "a/b.jpg", "c.png"); got != "c.png" {
t.Fatalf("got %q", got)
}
})
t.Run("falls back to photo.jpg when no candidate", func(t *testing.T) {
if got := safeMediaFileName(""); got != "photo.jpg" {
t.Fatalf("got %q", got)
}
})
t.Run("rejects slashed traversal candidate", func(t *testing.T) {
// Candidates containing "/" are rejected outright (not stripped), so a
// path-traversal candidate falls back to the safe default name.
if got := safeMediaFileName("../../x.png"); got != "photo.jpg" {
t.Fatalf("got %q", got)
}
})
t.Run("rejects dotdot candidate", func(t *testing.T) {
if got := safeMediaFileName(".."); got != "photo.jpg" {
t.Fatalf("got %q", got)
}
})
}
func TestExtensionFromContentTypes(t *testing.T) {
if got := extensionFromContentTypes("application/x-unknown-xyz"); got != ".jpg" {
t.Fatalf("expected .jpg fallback, got %q", got)
}
if got := extensionFromContentTypes("image/png"); !strings.HasPrefix(got, ".") {
t.Fatalf("expected an extension, got %q", got)
}
}
func TestValidateRemoteMediaURLSyntax(t *testing.T) {
t.Run("rejects non-http scheme", func(t *testing.T) {
if err := validateRemoteMediaURLSyntax("ftp://example.com/x"); err == nil {
t.Fatal("expected error for ftp scheme")
}
})
t.Run("rejects missing host", func(t *testing.T) {
if err := validateRemoteMediaURLSyntax("http://"); err == nil {
t.Fatal("expected error for missing host")
}
})
t.Run("allows http syntax", func(t *testing.T) {
if err := validateRemoteMediaURLSyntax("https://8.8.8.8/photo.jpg"); err != nil {
t.Fatalf("unexpected error: %v", err)
}
})
}
func TestPhotoFile(t *testing.T) {
ctx := context.Background()
t.Run("empty url", func(t *testing.T) {
photo := pluginsystem.Photo{Source: pluginsystem.MediaSource{Type: "url"}}
if _, _, err := photoFile(ctx, photo, Options{}, 1024); err == nil {
t.Fatal("expected error for empty url")
}
})
t.Run("unsupported type", func(t *testing.T) {
photo := pluginsystem.Photo{Source: pluginsystem.MediaSource{Type: "carrier"}}
if _, _, err := photoFile(ctx, photo, Options{}, 1024); err == nil {
t.Fatal("expected error for unsupported source type")
}
})
}
func TestPluginMediaBudgetRemainingBytes(t *testing.T) {
budget := &pluginMediaBudget{}
if got := budget.remainingBytes(); got != util.DefaultPluginMediaMaxBytes {
t.Fatalf("got %d, want per-file limit %d", got, util.DefaultPluginMediaMaxBytes)
}
budget.bytes = util.DefaultPluginMaxImportMediaBytes - 10
if got := budget.remainingBytes(); got != 10 {
t.Fatalf("got %d, want remaining aggregate budget", got)
}
budget.bytes = util.DefaultPluginMaxImportMediaBytes
if got := budget.remainingBytes(); got != 0 {
t.Fatalf("got %d, want exhausted budget", got)
}
}
func TestRemoveRawQueryParamOrdered(t *testing.T) {
raw := "z=last&api_key=secret&a=first&api_key=second"
if got := removeRawQueryParamOrdered(raw, "api_key"); got != "z=last&a=first" {
t.Fatalf("unexpected query: %q", got)
}
}

38
db/pluginsystem/auth_fields.go Normal file
View File

@@ -0,0 +1,38 @@
package pluginsystem
const (
AuthFieldAccessToken = "accessToken"
AuthFieldRefreshToken = "refreshToken"
AuthFieldClientSecret = "clientSecret"
AuthFieldOAuthState = "oauthState"
AuthFieldOAuthCodeVerifier = "oauthCodeVerifier"
AuthFieldOAuthRedirectURI = "oauthRedirectURI"
)
func InternalAuthSecretFields() []string {
return []string{
AuthFieldAccessToken,
AuthFieldRefreshToken,
AuthFieldClientSecret,
AuthFieldOAuthState,
AuthFieldOAuthCodeVerifier,
}
}
func InternalOAuthTransientFields() []string {
return []string{
AuthFieldOAuthState,
AuthFieldOAuthCodeVerifier,
AuthFieldOAuthRedirectURI,
}
}
func PluginInputAuthBlockedFields() []string {
return []string{
AuthFieldRefreshToken,
AuthFieldClientSecret,
AuthFieldOAuthState,
AuthFieldOAuthCodeVerifier,
AuthFieldOAuthRedirectURI,
}
}

349
db/pluginsystem/auth_injection.go Normal file
View File

@@ -0,0 +1,349 @@
package pluginsystem
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"github.com/pocketbase/pocketbase/core"
)
type AuthInjectionInput struct {
App core.App
Runtime Runtime
Session RuntimeSession
Plugin LocalPlugin
Instance *core.Record
Auth map[string]any
Config map[string]any
Spec *HostRequestSpec
Policy RequestPolicyContext
}
func InjectRequestAuthForContext(manifest Manifest, auth map[string]any, contextName string, req *http.Request) error {
if contextName == "" {
return nil
}
if err := ValidateAuthReference(manifest, contextName); err != nil {
return err
}
authContext, ok := manifest.Auth.Contexts[contextName]
if !ok {
return fmt.Errorf("plugin requested unknown auth context")
}
switch authContext.Type {
case AuthTypeOAuth2:
token := StringFromAny(auth[AuthFieldAccessToken])
if token == "" {
return fmt.Errorf("oauth access token is missing")
}
scheme := StringFromAny(auth[AuthFieldTokenType])
if scheme == "" {
scheme = AuthSchemeBearer
}
req.Header.Set(AuthHeaderAuthorization, scheme+" "+token)
case AuthTypeAPIKey:
secret := StringFromAny(auth[authContext.SecretField])
if secret == "" {
return fmt.Errorf("api key is missing")
}
name := authContext.Name
if name == "" {
name = authContext.SecretField
}
if authContext.Placement == AuthPlacementQuery {
req.URL.RawQuery = setRawQueryParamOrdered(req.URL.RawQuery, name, secret)
} else {
req.Header.Set(name, secret)
}
case AuthTypeBearer:
secret := StringFromAny(auth[authContext.SecretField])
if secret == "" {
return fmt.Errorf("bearer token is missing")
}
req.Header.Set(AuthHeaderAuthorization, AuthSchemeBearer+" "+secret)
default:
return fmt.Errorf("auth context %q is not supported for media requests", contextName)
}
return nil
}
func InjectHostRequestAuthFromPolicy(manifest Manifest, auth map[string]any, spec *HostRequestSpec) error {
if spec == nil || spec.Auth == "" {
return nil
}
if err := ValidateAuthReference(manifest, spec.Auth); err != nil {
return err
}
authContext, ok := manifest.Auth.Contexts[spec.Auth]
if !ok {
return fmt.Errorf("plugin requested unknown auth context")
}
switch authContext.Type {
case AuthTypeOAuth2:
token := StringFromAny(auth[AuthFieldAccessToken])
if token == "" {
return fmt.Errorf("oauth access token is missing")
}
scheme := StringFromAny(auth[AuthFieldTokenType])
if scheme == "" {
scheme = AuthSchemeBearer
}
setAuthHeader(spec, scheme+" "+token)
case AuthTypeAPIKey:
return injectAPIKeyAuth(authContext, auth, spec)
case AuthTypeBearer:
return injectBearerAuth(authContext, auth, spec)
case AuthTypeSession:
return fmt.Errorf("session auth requires handler-managed injection")
default:
return fmt.Errorf("auth context is not supported for host requests")
}
return nil
}
type pluginSessionResponse struct {
Token string `json:"token"`
Scheme string `json:"scheme,omitempty"`
Expires string `json:"expiresAt,omitempty"`
}
// ValidateAuthContext checks that a manifest auth context contains enough data
// for the host to own OAuth/API key/session injection safely.
func ValidateAuthContext(name string, context AuthContext) error {
switch context.Type {
case AuthTypeOAuth2:
if context.AuthorizationURL == "" || context.TokenURL == "" {
return fmt.Errorf("oauth2 auth context %s requires authorizationUrl and tokenUrl", name)
}
if _, err := url.ParseRequestURI(context.AuthorizationURL); err != nil {
return fmt.Errorf("auth context %s authorizationUrl: %w", name, err)
}
if _, err := url.ParseRequestURI(context.TokenURL); err != nil {
return fmt.Errorf("auth context %s tokenUrl: %w", name, err)
}
if context.Refresh == nil || context.Refresh.Mode != AuthRefreshModeHost {
return fmt.Errorf("oauth2 auth context %s must use host refresh", name)
}
case AuthTypeAPIKey, AuthTypeBearer:
if context.SecretField == "" {
return fmt.Errorf("%s auth context %s requires secretField", context.Type, name)
}
case AuthTypeSession:
if context.Refresh == nil || context.Refresh.Mode != AuthRefreshModePlugin || context.Refresh.Function == "" {
return fmt.Errorf("session auth context %s requires plugin refresh function", name)
}
if len(context.SecretFields) == 0 {
return fmt.Errorf("session auth context %s requires secretFields", name)
}
default:
return fmt.Errorf("auth context %s has unsupported type %q", name, context.Type)
}
return nil
}
// InjectHostRequestAuth resolves the auth reference from a HostRequestSpec and
// mutates the request with the provider-specific header/query/session token.
func InjectHostRequestAuth(ctx context.Context, input AuthInjectionInput) error {
if input.Spec == nil {
return fmt.Errorf("host request spec is required")
}
if input.Spec.Auth == "" {
return nil
}
if err := ValidateAuthReference(input.Plugin.Manifest, input.Spec.Auth); err != nil {
return err
}
authContext, ok := input.Plugin.Manifest.Auth.Contexts[input.Spec.Auth]
if !ok {
return fmt.Errorf("plugin requested unknown auth context")
}
switch authContext.Type {
case AuthTypeOAuth2:
return injectOAuthAuth(ctx, input, input.Spec.Auth)
case AuthTypeAPIKey:
return injectAPIKeyAuth(authContext, input.Auth, input.Spec)
case AuthTypeBearer:
return injectBearerAuth(authContext, input.Auth, input.Spec)
case AuthTypeSession:
return injectSessionAuth(ctx, input, authContext)
default:
return fmt.Errorf("auth context is not supported for route sending")
}
}
func injectOAuthAuth(ctx context.Context, input AuthInjectionInput, contextName string) error {
if input.Instance == nil {
return fmt.Errorf("plugin instance is required")
}
auth := input.Auth
if OAuthNeedsRefresh(auth) {
refreshed, err := RefreshOAuthToken(ctx, input.App, input.Plugin, input.Instance, auth, contextName)
if err != nil {
return fmt.Errorf("oauth token refresh failed: %w", err)
}
auth = refreshed
}
token := StringFromAny(auth[AuthFieldAccessToken])
if token == "" {
return fmt.Errorf("oauth access token is missing")
}
scheme := StringFromAny(auth[AuthFieldTokenType])
if scheme == "" {
scheme = AuthSchemeBearer
}
setAuthHeader(input.Spec, scheme+" "+token)
return nil
}
func injectAPIKeyAuth(authContext AuthContext, auth map[string]any, spec *HostRequestSpec) error {
secret := StringFromAny(auth[authContext.SecretField])
if secret == "" {
return fmt.Errorf("api key is missing")
}
if authContext.Placement == AuthPlacementQuery {
name := authContext.Name
if name == "" {
name = authContext.SecretField
}
query := make([]QueryParam, 0, len(spec.Target.Query)+1)
for _, param := range spec.Target.Query {
if param.Name != name {
query = append(query, param)
}
}
query = append(query, QueryParam{Name: name, Value: secret})
spec.Target.Query = query
return nil
}
name := authContext.Name
if name == "" {
name = AuthHeaderAuthorization
}
if spec.Headers == nil {
spec.Headers = map[string]string{}
}
spec.Headers[name] = secret
return nil
}
func injectBearerAuth(authContext AuthContext, auth map[string]any, spec *HostRequestSpec) error {
secret := StringFromAny(auth[authContext.SecretField])
if secret == "" {
return fmt.Errorf("bearer token is missing")
}
setAuthHeader(spec, AuthSchemeBearer+" "+secret)
return nil
}
func injectSessionAuth(ctx context.Context, input AuthInjectionInput, authContext AuthContext) error {
if authContext.Refresh == nil || authContext.Refresh.Mode != AuthRefreshModePlugin {
return fmt.Errorf("session auth context is not supported")
}
if input.Instance == nil {
return fmt.Errorf("plugin instance is required")
}
pluginInput := map[string]any{
"instance": InstanceRef{
ID: input.Instance.Id,
PluginID: input.Instance.GetString("plugin_id"),
},
"auth": AuthForPluginRefresh(input.Auth, authContext),
"config": input.Config,
}
inputBytes, err := json.Marshal(pluginInput)
if err != nil {
return err
}
var output []byte
if input.Session != nil {
output, err = input.Session.Call(ctx, authContext.Refresh.Function, inputBytes)
} else {
output, err = input.Runtime.Call(ctx, input.Plugin, authContext.Refresh.Function, inputBytes, input.Policy)
}
if err != nil {
return err
}
var session pluginSessionResponse
if err := validatePluginSessionRefreshOutput(output, &session); err != nil {
return err
}
scheme := session.Scheme
if scheme == "" {
scheme = AuthSchemeBearer
}
setAuthHeader(input.Spec, scheme+" "+session.Token)
return nil
}
func ValidatePluginSessionRefreshOutput(output []byte) error {
var session pluginSessionResponse
return validatePluginSessionRefreshOutput(output, &session)
}
func validatePluginSessionRefreshOutput(output []byte, session *pluginSessionResponse) error {
if err := json.Unmarshal(output, session); err != nil {
return fmt.Errorf("plugin returned an invalid session: %w", err)
}
if session.Token == "" {
return fmt.Errorf("plugin returned an empty session token")
}
return nil
}
func setAuthHeader(spec *HostRequestSpec, value string) {
if spec.Headers == nil {
spec.Headers = map[string]string{}
}
spec.Headers[AuthHeaderAuthorization] = value
}
func setRawQueryParamOrdered(rawQuery string, name string, value string) string {
encoded := url.QueryEscape(name) + "=" + url.QueryEscape(value)
if rawQuery == "" {
return encoded
}
parts := strings.Split(rawQuery, "&")
kept := make([]string, 0, len(parts)+1)
for _, part := range parts {
if part == "" {
continue
}
rawName := part
if idx := strings.Index(rawName, "="); idx >= 0 {
rawName = rawName[:idx]
}
decodedName, err := url.QueryUnescape(rawName)
if err == nil && decodedName == name {
continue
}
kept = append(kept, part)
}
kept = append(kept, encoded)
return strings.Join(kept, "&")
}
func AuthForPluginRefresh(auth map[string]any, authContext AuthContext) map[string]any {
filtered := map[string]any{}
for _, field := range authContext.Fields {
if value, ok := auth[field]; ok {
filtered[field] = value
}
}
for _, field := range authContext.SecretFields {
if value, ok := auth[field]; ok {
filtered[field] = value
}
}
if authContext.SecretField != "" {
if value, ok := auth[authContext.SecretField]; ok {
filtered[authContext.SecretField] = value
}
}
return filtered
}

288
db/pluginsystem/auth_injection_test.go Normal file
View File

@@ -0,0 +1,288 @@
package pluginsystem
import (
"context"
"net/http"
"testing"
)
func TestValidateAuthContext(t *testing.T) {
tests := []struct {
name string
context AuthContext
wantErr bool
}{
{
name: "oauth2",
context: AuthContext{
Type: AuthTypeOAuth2,
AuthorizationURL: "https://example.com/oauth/authorize",
TokenURL: "https://example.com/oauth/token",
Refresh: &AuthRefresh{Mode: AuthRefreshModeHost},
},
},
{
name: "missing bearer secret",
context: AuthContext{Type: AuthTypeBearer},
wantErr: true,
},
{
name: "session",
context: AuthContext{
Type: AuthTypeSession,
SecretFields: []string{"email", "password"},
Refresh: &AuthRefresh{Mode: AuthRefreshModePlugin, Function: "refresh_session_v1"},
},
},
{
name: "unsupported",
context: AuthContext{Type: "mtls"},
wantErr: true,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
err := ValidateAuthContext("default", test.context)
if test.wantErr && err == nil {
t.Fatal("expected error")
}
if !test.wantErr && err != nil {
t.Fatalf("unexpected error: %v", err)
}
})
}
}
func TestInjectHostRequestAuthWithBearer(t *testing.T) {
spec := HostRequestSpec{Auth: "account"}
err := InjectHostRequestAuth(context.Background(), AuthInjectionInput{
Plugin: LocalPlugin{Manifest: Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {
Type: AuthTypeBearer,
SecretField: "token",
},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}},
Auth: map[string]any{"token": "abc123"},
Spec: &spec,
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got := spec.Headers[AuthHeaderAuthorization]; got != AuthSchemeBearer+" abc123" {
t.Fatalf("unexpected authorization header: %q", got)
}
}
func TestInjectHostRequestAuthWithAPIKeyQuery(t *testing.T) {
spec := HostRequestSpec{
Auth: "account",
Target: RequestTarget{
Type: "connector",
Connector: "api",
Path: "/upload",
Query: []QueryParam{{Name: "existing", Value: "true"}},
},
}
err := InjectHostRequestAuth(context.Background(), AuthInjectionInput{
Plugin: LocalPlugin{Manifest: Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {
Type: AuthTypeAPIKey,
SecretField: "apiKey",
Placement: AuthPlacementQuery,
Name: "key",
},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}},
Auth: map[string]any{"apiKey": "secret"},
Spec: &spec,
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(spec.Target.Query) != 2 || spec.Target.Query[1].Name != "key" || spec.Target.Query[1].Value != "secret" {
t.Fatalf("unexpected query: %#v", spec.Target.Query)
}
}
func TestInjectHostRequestAuthFromPolicyWithBearer(t *testing.T) {
spec := HostRequestSpec{
Auth: "account",
Headers: map[string]string{AuthHeaderAuthorization: "plugin supplied"},
}
manifest := Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {Type: AuthTypeBearer, SecretField: "token"},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}
err := InjectHostRequestAuthFromPolicy(manifest, map[string]any{"token": "host-secret"}, &spec)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got := spec.Headers[AuthHeaderAuthorization]; got != AuthSchemeBearer+" host-secret" {
t.Fatalf("unexpected authorization header: %q", got)
}
}
func TestInjectHostRequestAuthFromPolicyFailsWithEmptyAuth(t *testing.T) {
spec := HostRequestSpec{
Auth: "account",
Headers: map[string]string{AuthHeaderAuthorization: "plugin supplied"},
}
manifest := Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {Type: AuthTypeBearer, SecretField: "token"},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}
err := InjectHostRequestAuthFromPolicy(manifest, map[string]any{}, &spec)
if err == nil {
t.Fatal("expected error")
}
if err.Error() != "bearer token is missing" {
t.Fatalf("unexpected error: %v", err)
}
if got := spec.Headers[AuthHeaderAuthorization]; got != "plugin supplied" {
t.Fatalf("unexpected authorization header mutation: %q", got)
}
}
func TestInjectHostRequestAuthFromPolicyRejectsSessionAuth(t *testing.T) {
spec := HostRequestSpec{Auth: "account"}
manifest := Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {
Type: AuthTypeSession,
SecretFields: []string{"password"},
Refresh: &AuthRefresh{Mode: AuthRefreshModePlugin, Function: "refresh_session_v1"},
},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}
err := InjectHostRequestAuthFromPolicy(manifest, map[string]any{"password": "secret"}, &spec)
if err == nil {
t.Fatal("expected error")
}
if err.Error() != "session auth requires handler-managed injection" {
t.Fatalf("unexpected error: %v", err)
}
}
func TestInjectHostRequestAuthFromPolicyWithAPIKeyQuery(t *testing.T) {
spec := HostRequestSpec{
Auth: "account",
Target: RequestTarget{
Type: "connector",
Path: "/assets",
Query: []QueryParam{{Name: "api_key", Value: "plugin"}},
},
}
manifest := Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {
Type: AuthTypeAPIKey,
SecretField: "apiKey",
Placement: AuthPlacementQuery,
Name: "api_key",
},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}
err := InjectHostRequestAuthFromPolicy(manifest, map[string]any{"apiKey": "host-secret"}, &spec)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(spec.Target.Query) != 1 || spec.Target.Query[0].Value != "host-secret" {
t.Fatalf("unexpected query: %#v", spec.Target.Query)
}
}
func TestInjectHostRequestAuthRequiresSpec(t *testing.T) {
err := InjectHostRequestAuth(context.Background(), AuthInjectionInput{})
if err == nil {
t.Fatal("expected error")
}
if err.Error() != "host request spec is required" {
t.Fatalf("unexpected error: %v", err)
}
}
func TestInjectHostRequestAuthValidatesPermission(t *testing.T) {
spec := HostRequestSpec{Auth: "account"}
err := InjectHostRequestAuth(context.Background(), AuthInjectionInput{
Plugin: LocalPlugin{Manifest: Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {Type: AuthTypeBearer, SecretField: "token"},
}},
}},
Auth: map[string]any{"token": "abc123"},
Spec: &spec,
})
if err == nil {
t.Fatal("expected error")
}
if err.Error() != `auth context "account" is not permitted` {
t.Fatalf("unexpected error: %v", err)
}
}
func TestInjectRequestAuthForContextPreservesQueryOrder(t *testing.T) {
req, err := http.NewRequest(http.MethodGet, "https://example.test/media?z=last&api_key=plugin&a=first", nil)
if err != nil {
t.Fatal(err)
}
manifest := Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"account": {
Type: AuthTypeAPIKey,
SecretField: "apiKey",
Placement: AuthPlacementQuery,
Name: "api_key",
},
}},
Permissions: PermissionManifest{Auth: []string{"account"}},
}
err = InjectRequestAuthForContext(manifest, map[string]any{"apiKey": "host-secret"}, "account", req)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if req.URL.RawQuery != "z=last&a=first&api_key=host-secret" {
t.Fatalf("unexpected raw query: %q", req.URL.RawQuery)
}
}
func TestAuthForPluginRefresh(t *testing.T) {
filtered := AuthForPluginRefresh(map[string]any{
"email": "user@example.com",
"password": "secret",
"accessToken": "token",
}, AuthContext{
Fields: []string{"email", "password"},
SecretFields: []string{"password"},
})
if len(filtered) != 2 {
t.Fatalf("unexpected filtered auth: %#v", filtered)
}
if filtered["email"] != "user@example.com" || filtered["password"] != "secret" {
t.Fatalf("unexpected filtered auth: %#v", filtered)
}
if _, ok := filtered["accessToken"]; ok {
t.Fatalf("unexpected access token in plugin refresh auth: %#v", filtered)
}
}

410
db/pluginsystem/host_http.go Normal file
View File

@@ -0,0 +1,410 @@
package pluginsystem
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"log"
"mime"
"mime/multipart"
"net/http"
"net/url"
"strings"
"pocketbase/util"
extism "github.com/extism/go-sdk"
)
type hostHTTPResponse struct {
Status int `json:"status"`
HeaderValues map[string][]string `json:"headerValues,omitempty"`
BodyBase64 string `json:"bodyBase64,omitempty"`
Error *PluginError `json:"error,omitempty"`
}
type HostRequestOptions struct {
Trail []byte
}
type HostResponse struct {
Status int
HeaderValues map[string][]string
Body []byte
}
var newConnectorHTTPClient = util.ConnectorHTTPClient
const maxHostLogPayloadBytes = 8 * 1024
// extismHostFunctions exposes the host APIs that WASM plugins may call. Each
// function must delegate to the same policy-controlled host implementation that
// backend handlers use.
func extismHostFunctions(manifest Manifest, policy RequestPolicyContext) []extism.HostFunction {
httpFn := extism.NewHostFunctionWithStack(
"http_request",
func(ctx context.Context, plugin *extism.CurrentPlugin, stack []uint64) {
requestBytes, err := plugin.ReadBytes(stack[0])
if err != nil {
writeHostHTTPResponse(ctx, plugin, stack, hostHTTPResponse{
Error: &PluginError{Code: "invalid_request", Message: err.Error()},
})
return
}
response := executeHostHTTPRequest(ctx, manifest, policy, requestBytes)
writeHostHTTPResponse(ctx, plugin, stack, response)
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
httpFn.SetNamespace("wanderer")
logFn := extism.NewHostFunctionWithStack(
"log",
func(ctx context.Context, plugin *extism.CurrentPlugin, stack []uint64) {
message, err := readBoundedHostLogPayload(plugin, stack[0])
if err != nil {
plugin.Log(extism.LogLevelError, "read host log message: "+err.Error())
return
}
entry, err := parseHostLogEntry(message)
if err != nil {
plugin.Log(extism.LogLevelError, "invalid host log message: "+err.Error())
return
}
log.Printf("plugin log [%s]: %s", entry.Level, entry.Message)
_ = ctx
},
[]extism.ValueType{extism.ValueTypePTR},
nil,
)
logFn.SetNamespace("wanderer")
return []extism.HostFunction{httpFn, logFn}
}
func readBoundedHostLogPayload(plugin *extism.CurrentPlugin, offset uint64) ([]byte, error) {
length, err := plugin.Length(offset)
if err != nil {
return nil, err
}
if length > maxHostLogPayloadBytes {
return nil, fmt.Errorf("log message exceeds maximum size")
}
return plugin.ReadBytes(offset)
}
func parseHostLogEntry(message []byte) (HostLogEntry, error) {
if len(message) > maxHostLogPayloadBytes {
return HostLogEntry{}, fmt.Errorf("log message exceeds maximum size")
}
var entry HostLogEntry
if err := json.Unmarshal(message, &entry); err != nil {
return HostLogEntry{}, fmt.Errorf("decode log entry: %w", err)
}
level, err := normalizeHostLogLevel(entry.Level)
if err != nil {
return HostLogEntry{}, err
}
entry.Level = level
entry.Message = sanitizeHostLogMessage(entry.Message)
if entry.Message == "" {
return HostLogEntry{}, fmt.Errorf("log message is required")
}
return entry, nil
}
func sanitizeHostLogMessage(message string) string {
return strings.TrimSpace(strings.Map(func(r rune) rune {
if r < 0x20 || r == 0x7f {
return ' '
}
return r
}, message))
}
func normalizeHostLogLevel(level string) (string, error) {
switch strings.ToLower(strings.TrimSpace(level)) {
case "debug":
return "debug", nil
case "info":
return "info", nil
case "warn":
return "warn", nil
case "error":
return "error", nil
default:
return "", fmt.Errorf("unsupported log level %q", level)
}
}
// executeHostHTTPRequest turns a raw plugin http_request payload into the
// hostHTTPResponse that the plugin reads back. It is the single source of truth
// for the request/response contract shared by the in-process runtime
// (extismHostFunctions) and the worker process (handleHostHTTPRequest), so the
// two paths cannot drift on error codes or response shape.
func executeHostHTTPRequest(ctx context.Context, manifest Manifest, policy RequestPolicyContext, requestBytes []byte) hostHTTPResponse {
var spec HostRequestSpec
if err := json.Unmarshal(requestBytes, &spec); err != nil {
return hostHTTPResponse{
Error: &PluginError{Code: "invalid_request", Message: "invalid host request: " + err.Error()},
}
}
executed, err := ExecuteHostRequest(ctx, manifest, policy, spec, HostRequestOptions{})
if err != nil {
return hostHTTPResponse{
Error: &PluginError{Code: "provider_unavailable", Message: err.Error()},
}
}
return hostHTTPResponse{
Status: executed.Status,
HeaderValues: executed.HeaderValues,
BodyBase64: base64.StdEncoding.EncodeToString(executed.Body),
}
}
func writeHostHTTPResponse(ctx context.Context, plugin *extism.CurrentPlugin, stack []uint64, response hostHTTPResponse) {
responseBytes, err := json.Marshal(response)
if err != nil {
responseBytes, _ = json.Marshal(hostHTTPResponse{
Error: &PluginError{Code: "internal_error", Message: err.Error()},
})
}
offset, err := plugin.WriteBytes(responseBytes)
if err != nil {
plugin.Log(extism.LogLevelError, "write host http response: "+err.Error())
stack[0] = 0
return
}
stack[0] = offset
_ = ctx
}
// ExecuteHostRequest is the single network chokepoint for plugin-controlled
// HTTP. It validates manifest policy, builds optional request bodies, enforces
// upload/response limits, follows only permitted redirects, and returns the
// bounded provider response.
func ExecuteHostRequest(ctx context.Context, manifest Manifest, policy RequestPolicyContext, spec HostRequestSpec, options HostRequestOptions) (HostResponse, error) {
if err := InjectHostRequestAuthFromPolicy(manifest, policy.HostAuth, &spec); err != nil {
return HostResponse{}, err
}
resolved, err := ValidateAndResolveHostRequestSpec(manifest, spec, policy)
if err != nil {
return HostResponse{}, err
}
body, contentType, bodySize, err := hostRequestBody(spec, options)
if err != nil {
return HostResponse{}, err
}
if err := validateHostRequestUpload(manifest, spec, contentType, bodySize); err != nil {
return HostResponse{}, err
}
req, err := http.NewRequestWithContext(ctx, spec.Method, resolved.URL.String(), body)
if err != nil {
return HostResponse{}, err
}
for key, value := range spec.Headers {
req.Header.Set(key, value)
}
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
if req.Header.Get("Accept") == "" {
req.Header.Set("Accept", "application/json")
}
client, err := newConnectorHTTPClient(util.ConnectorHTTPPolicy{
BaseURL: resolved.Connector.BaseURL,
AllowPrivate: resolved.Connector.AllowPrivate,
TLSMode: resolved.Connector.TLS.Mode,
TLSCABundle: resolved.Connector.TLS.CABundle,
}, func(req *http.Request, via []*http.Request) error {
if spec.FollowRedirects != nil && !*spec.FollowRedirects {
return http.ErrUseLastResponse
}
if len(via) >= 10 {
return fmt.Errorf("too many redirects")
}
previous := resolved.URL
if len(via) > 0 {
previous = via[len(via)-1].URL
}
return ValidateConnectorRedirect(resolved.Connector, previous, req.URL)
})
if err != nil {
return HostResponse{}, err
}
resp, err := client.Do(req)
if err != nil {
return HostResponse{}, err
}
defer resp.Body.Close()
if err := validateHostHTTPResponse(manifest, spec, resp); err != nil {
return HostResponse{}, err
}
maxBytes := effectiveResponseMaxBytes(manifest, spec)
limit := maxBytes
if limit <= 0 {
limit = 1 << 20
}
bodyBytes, err := io.ReadAll(io.LimitReader(resp.Body, limit+1))
if err != nil {
return HostResponse{}, err
}
if maxBytes > 0 && int64(len(bodyBytes)) > maxBytes {
return HostResponse{}, fmt.Errorf("provider response exceeds maximum size")
}
if maxBytes <= 0 && int64(len(bodyBytes)) > limit {
return HostResponse{}, fmt.Errorf("provider response exceeds default maximum size")
}
headerValues := map[string][]string{}
for key, values := range resp.Header {
if len(values) > 0 {
headerValues[key] = append([]string{}, values...)
}
}
return HostResponse{
Status: resp.StatusCode,
HeaderValues: headerValues,
Body: bodyBytes,
}, nil
}
func hostRequestBody(spec HostRequestSpec, options HostRequestOptions) (io.Reader, string, int64, error) {
if spec.Body == nil {
return nil, "", 0, nil
}
switch spec.Body.Type {
case HostRequestBodyTypeJSON:
body, err := json.Marshal(spec.Body.JSON)
if err != nil {
return nil, "", 0, err
}
return bytes.NewReader(body), "application/json", int64(len(body)), nil
case HostRequestBodyTypeForm:
body, err := formURLEncodedBody(spec.Body.Form)
if err != nil {
return nil, "", 0, err
}
return strings.NewReader(body), "application/x-www-form-urlencoded", int64(len(body)), nil
case HostRequestBodyTypeMultipart:
var body bytes.Buffer
writer := multipart.NewWriter(&body)
for _, part := range spec.Body.Parts {
if part.Source == MultipartSourceTrail || part.Source == MultipartSourceTrailGPX {
if len(options.Trail) == 0 {
return nil, "", 0, fmt.Errorf("multipart part %q requires trail content", part.Name)
}
filename := part.Filename
if filename == "" {
filename = MultipartTrailFilename
}
partWriter, err := writer.CreateFormFile(part.Name, filename)
if err != nil {
return nil, "", 0, err
}
if _, err := partWriter.Write(options.Trail); err != nil {
return nil, "", 0, err
}
continue
}
if part.JSON != nil {
data, err := json.Marshal(part.JSON)
if err != nil {
return nil, "", 0, err
}
if err := writer.WriteField(part.Name, string(data)); err != nil {
return nil, "", 0, err
}
}
}
if err := writer.Close(); err != nil {
return nil, "", 0, err
}
return &body, writer.FormDataContentType(), int64(body.Len()), nil
default:
return nil, "", 0, fmt.Errorf("unsupported host request body type %q", spec.Body.Type)
}
}
func formURLEncodedBody(fields []FormField) (string, error) {
encoded := make([]string, 0, len(fields))
for _, field := range fields {
if field.Name == "" {
return "", fmt.Errorf("form field name must not be empty")
}
if hasControl(field.Name) || hasControl(field.Value) {
return "", fmt.Errorf("form fields must not contain control characters")
}
encoded = append(encoded, url.QueryEscape(field.Name)+"="+url.QueryEscape(field.Value))
}
return strings.Join(encoded, "&"), nil
}
func validateHostRequestUpload(manifest Manifest, spec HostRequestSpec, contentType string, bodySize int64) error {
if spec.Body == nil {
return nil
}
if manifest.Permissions.Uploads.MaxBytes > 0 && bodySize > manifest.Permissions.Uploads.MaxBytes {
return fmt.Errorf("host request upload exceeds manifest upload limit")
}
if contentType == "" || len(manifest.Permissions.Uploads.ContentTypes) == 0 {
return nil
}
mediaType, _, err := mime.ParseMediaType(contentType)
if err != nil {
return fmt.Errorf("host request upload has invalid content type")
}
for _, allowed := range manifest.Permissions.Uploads.ContentTypes {
if strings.EqualFold(mediaType, allowed) {
return nil
}
}
return fmt.Errorf("host request upload content type %q is not allowed", mediaType)
}
func validateHostHTTPResponse(manifest Manifest, spec HostRequestSpec, resp *http.Response) error {
allowedContentTypes := effectiveResponseContentTypes(manifest, spec)
if resp.StatusCode >= 200 && resp.StatusCode < 300 && len(allowedContentTypes) > 0 {
contentType := resp.Header.Get("Content-Type")
mediaType, _, err := mime.ParseMediaType(contentType)
if err != nil || mediaType == "" {
return fmt.Errorf("provider response has invalid content type")
}
allowed := false
for _, expected := range allowedContentTypes {
if strings.EqualFold(mediaType, expected) {
allowed = true
break
}
}
if !allowed {
return fmt.Errorf("provider response content type %q is not allowed", mediaType)
}
}
maxBytes := effectiveResponseMaxBytes(manifest, spec)
if maxBytes > 0 && resp.ContentLength > maxBytes {
return fmt.Errorf("provider response exceeds maximum size")
}
return nil
}
func effectiveResponseContentTypes(manifest Manifest, spec HostRequestSpec) []string {
if len(spec.Expect.ContentTypes) > 0 {
return spec.Expect.ContentTypes
}
return manifest.Permissions.Downloads.ContentTypes
}
func effectiveResponseMaxBytes(manifest Manifest, spec HostRequestSpec) int64 {
if spec.Expect.MaxBytes > 0 {
return spec.Expect.MaxBytes
}
return manifest.Permissions.Downloads.MaxBytes
}

407
db/pluginsystem/host_http_test.go Normal file
View File

@@ -0,0 +1,407 @@
package pluginsystem
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"pocketbase/util"
)
func TestParseHostLogEntry(t *testing.T) {
payload, err := json.Marshal(HostLogEntry{Level: "warn", Message: " slow request "})
if err != nil {
t.Fatal(err)
}
entry, err := parseHostLogEntry(payload)
if err != nil {
t.Fatal(err)
}
if entry.Level != "warn" || entry.Message != "slow request" {
t.Fatalf("unexpected structured entry: %#v", entry)
}
if _, err := parseHostLogEntry([]byte(" plain message ")); err == nil {
t.Fatal("expected plain log message to fail")
}
if _, err := parseHostLogEntry([]byte(`{"level":"verbose","message":"hello"}`)); err == nil {
t.Fatal("expected unsupported log level to fail")
}
if _, err := parseHostLogEntry([]byte(`{"level":"info","message":" "}`)); err == nil {
t.Fatal("expected empty log message to fail")
}
}
func TestParseHostLogEntrySanitizesMessage(t *testing.T) {
entry, err := parseHostLogEntry([]byte(`{"level":"info","message":"first\nsecond\rthird\tfourth"}`))
if err != nil {
t.Fatal(err)
}
if entry.Message != "first second third fourth" {
t.Fatalf("unexpected sanitized message: %q", entry.Message)
}
}
func TestParseHostLogEntryRejectsOversizedPayload(t *testing.T) {
payload := []byte(`{"level":"info","message":"` + strings.Repeat("x", maxHostLogPayloadBytes) + `"}`)
if _, err := parseHostLogEntry(payload); err == nil {
t.Fatal("expected oversized log payload to fail")
}
}
func TestExecuteHostRequestRejectsRedirectToUndeclaredHost(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "https://evil.example.test/v1/upload", http.StatusFound)
}))
defer server.Close()
_, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "GET",
Target: RequestTarget{
Type: "connector",
Connector: "api",
Path: "/v1",
},
}, HostRequestOptions{})
if err == nil {
t.Fatal("expected redirect policy error")
}
}
func TestExecuteHostRequestRejectsRedirectOutsidePathScope(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/admin", http.StatusFound)
}))
defer server.Close()
_, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1"},
}, HostRequestOptions{})
if err == nil {
t.Fatal("expected redirect policy error")
}
}
func TestExecuteHostRequestEnforcesResponseLimit(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"too":"large"}`))
}))
defer server.Close()
_, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1"},
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 4,
},
}, HostRequestOptions{})
if err == nil {
t.Fatal("expected maxBytes error")
}
}
func TestExecuteHostRequestAllowsErrorResponseWithoutContentType(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusUnauthorized)
_, _ = w.Write([]byte(`missing credentials`))
}))
defer server.Close()
resp, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1"},
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 1024,
},
}, HostRequestOptions{})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Status != http.StatusUnauthorized || string(resp.Body) != "missing credentials" {
t.Fatalf("unexpected response: %#v body=%q", resp, string(resp.Body))
}
}
func TestExecuteHostRequestInjectsAPIKeyQueryBeforeBuildingURL(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if got := r.URL.Query().Get("api_key"); got != "host-secret" {
t.Fatalf("api_key = %q, want host-secret; raw query %q", got, r.URL.RawQuery)
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true}`))
}))
defer server.Close()
manifest := testHostManifest(t, server.URL)
manifest.Auth = AuthManifest{Contexts: map[string]AuthContext{
"account": {
Type: AuthTypeAPIKey,
SecretField: "apiKey",
Placement: AuthPlacementQuery,
Name: "api_key",
},
}}
manifest.Permissions.Auth = []string{"account"}
manifest.Permissions.Network.Connectors[0].Auth = []string{"account"}
policy := testHostPolicy(t, server.URL).WithHostAuth(map[string]any{"apiKey": "host-secret"})
policy.Connectors["api"] = ResolvedConnectorTarget{
Name: "api",
Type: ConnectorTypePublicAPI,
BaseURL: policy.Connectors["api"].BaseURL,
BasePath: "/",
AllowPrivate: true,
AllowedPathPrefixes: []string{"/v1"},
Auth: []string{"account"},
}
resp, err := ExecuteHostRequest(context.Background(), manifest, policy, HostRequestSpec{
Method: "GET",
Auth: "account",
Target: RequestTarget{
Type: "connector",
Connector: "api",
Path: "/v1",
Query: []QueryParam{{Name: "existing", Value: "1"}},
},
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 1024,
},
}, HostRequestOptions{})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Status != http.StatusOK {
t.Fatalf("unexpected status %d", resp.Status)
}
}
func TestExecuteHostRequestBuildsMultipartTrailSend(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if mediaType := strings.Split(r.Header.Get("Content-Type"), ";")[0]; mediaType != "multipart/form-data" {
t.Fatalf("unexpected content type %q", r.Header.Get("Content-Type"))
}
file, header, err := r.FormFile("file")
if err != nil {
t.Fatalf("expected file part: %v", err)
}
defer file.Close()
if header.Filename != "My Route.gpx" {
t.Fatalf("unexpected filename %q", header.Filename)
}
data, _ := io.ReadAll(file)
if string(data) != "<gpx />" {
t.Fatalf("unexpected trail body %q", string(data))
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true}`))
}))
defer server.Close()
resp, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "POST",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1/upload"},
Body: &HostRequestBody{
Type: HostRequestBodyTypeMultipart,
Parts: []MultipartPart{{
Name: "file",
Source: MultipartSourceTrail,
Filename: "My Route.gpx",
}},
},
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 1024,
},
}, HostRequestOptions{Trail: []byte("<gpx />")})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Status != http.StatusOK {
t.Fatalf("unexpected status %d", resp.Status)
}
}
func TestExecuteHostRequestBuildsFormURLEncodedBody(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if got := r.Header.Get("Content-Type"); got != "application/x-www-form-urlencoded" {
t.Fatalf("unexpected content type %q", got)
}
if err := r.ParseForm(); err != nil {
t.Fatalf("parse form: %v", err)
}
if got := r.Form.Get("person[login_identity]"); got != "user@example.test" {
t.Fatalf("login_identity = %q", got)
}
if got := r.Form.Get("person[password]"); got != "secret" {
t.Fatalf("password = %q", got)
}
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true}`))
}))
defer server.Close()
manifest := testHostManifest(t, server.URL)
manifest.Permissions.Uploads.ContentTypes = append(manifest.Permissions.Uploads.ContentTypes, "application/x-www-form-urlencoded")
resp, err := ExecuteHostRequest(context.Background(), manifest, testHostPolicy(t, server.URL), HostRequestSpec{
Method: "POST",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1/login"},
Body: &HostRequestBody{
Type: HostRequestBodyTypeForm,
Form: []FormField{
{Name: "person[login_identity]", Value: "user@example.test"},
{Name: "person[password]", Value: "secret"},
},
},
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 1024,
},
}, HostRequestOptions{})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Status != http.StatusOK {
t.Fatalf("unexpected status %d", resp.Status)
}
}
func TestExecuteHostRequestCanReturnRedirectResponse(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/v1/next", http.StatusFound)
}))
defer server.Close()
followRedirects := false
resp, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1/start"},
FollowRedirects: &followRedirects,
}, HostRequestOptions{})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if resp.Status != http.StatusFound {
t.Fatalf("unexpected status %d", resp.Status)
}
if got := resp.HeaderValues["Location"]; len(got) != 1 || got[0] != "/v1/next" {
t.Fatalf("Location = %#v", got)
}
}
func TestExecuteHostRequestReturnsMultiValueHeaders(t *testing.T) {
useUnsafeTestHTTPClient(t)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Add("Set-Cookie", "session=abc; Path=/")
w.Header().Add("Set-Cookie", "device=full; Path=/")
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true}`))
}))
defer server.Close()
resp, err := ExecuteHostRequest(context.Background(), testHostManifest(t, server.URL), testHostPolicy(t, server.URL), HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1"},
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 1024,
},
}, HostRequestOptions{})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if got := resp.HeaderValues["Set-Cookie"]; len(got) != 2 || got[0] != "session=abc; Path=/" || got[1] != "device=full; Path=/" {
t.Fatalf("Set-Cookie values = %#v", got)
}
}
func useUnsafeTestHTTPClient(t *testing.T) {
t.Helper()
original := newConnectorHTTPClient
newConnectorHTTPClient = func(policy util.ConnectorHTTPPolicy, checkRedirect func(req *http.Request, via []*http.Request) error) (*http.Client, error) {
return &http.Client{
Timeout: 60 * time.Second,
CheckRedirect: checkRedirect,
}, nil
}
t.Cleanup(func() {
newConnectorHTTPClient = original
})
}
func testHostManifest(t *testing.T, rawURL string) Manifest {
t.Helper()
return Manifest{
ManifestVersion: ManifestVersion,
ID: "test",
Type: PluginTypeTrails,
Name: "Test",
Version: "0.1.0",
Runtime: RuntimeManifest{
Type: RuntimeWASM,
Entrypoint: "plugin.wasm",
},
Capabilities: []CapabilityManifest{{
Name: "test",
Version: "v1",
Export: "test_v1",
}},
Permissions: PermissionManifest{
Network: NetworkPermissions{
Connectors: []ConnectorTargetPermission{{
Name: "api",
Type: ConnectorTypePublicAPI,
FixedBaseURL: rawURL,
AllowedPathPrefixes: []string{"/v1"},
}},
},
Downloads: DownloadPermissions{
MaxBytes: 1024,
ContentTypes: []string{"application/json"},
},
Uploads: UploadPermissions{
MaxBytes: 1024,
ContentTypes: []string{"multipart/form-data"},
},
},
}
}
func testHostPolicy(t *testing.T, rawURL string) RequestPolicyContext {
t.Helper()
parsed, err := url.Parse(rawURL)
if err != nil {
t.Fatal(err)
}
parsed.Path = ""
return RequestPolicyContext{Connectors: map[string]ResolvedConnectorTarget{
"api": {
Name: "api",
Type: ConnectorTypePublicAPI,
BaseURL: parsed.String(),
BasePath: "/",
AllowPrivate: true,
AllowedPathPrefixes: []string{"/v1"},
},
}}
}

75
db/pluginsystem/import_types.go Normal file
View File

@@ -0,0 +1,75 @@
package pluginsystem
import "time"
type InstanceRef struct {
ID string `json:"id"`
PluginID string `json:"pluginId"`
}
type TrailImport struct {
Source TrailImportSource `json:"source"`
Kind string `json:"kind,omitempty"`
Name string `json:"name"`
Description string `json:"description,omitempty"`
StartedAt *time.Time `json:"startedAt,omitempty"`
ActivityType string `json:"activityType,omitempty"`
Privacy *string `json:"privacy,omitempty"`
Track Track `json:"track"`
Waypoints []Waypoint `json:"waypoints,omitempty"`
Photos []Photo `json:"photos,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type TrailSummary struct {
Source TrailImportSource `json:"source"`
Kind string `json:"kind,omitempty"`
}
type TrailImportSource struct {
Provider string `json:"provider"`
ExternalID string `json:"externalId"`
URL string `json:"url,omitempty"`
}
type Track struct {
Format string `json:"format"`
ContentBase64 string `json:"contentBase64"`
}
type Waypoint struct {
ExternalID string `json:"externalId,omitempty"`
Name string `json:"name,omitempty"`
Description string `json:"description,omitempty"`
Lat float64 `json:"lat"`
Lon float64 `json:"lon"`
Ele *float64 `json:"ele,omitempty"`
Time *time.Time `json:"time,omitempty"`
Icon string `json:"icon,omitempty"`
Photos []Photo `json:"photos,omitempty"`
}
type Photo struct {
ExternalID string `json:"externalId,omitempty"`
Filename string `json:"filename,omitempty"`
ContentType string `json:"contentType,omitempty"`
TakenAt *time.Time `json:"takenAt,omitempty"`
Lat *float64 `json:"lat,omitempty"`
Lon *float64 `json:"lon,omitempty"`
Source MediaSource `json:"source"`
}
type MediaSource struct {
Type string `json:"type"`
URL string `json:"url,omitempty"`
MediaRef *MediaRef `json:"mediaRef,omitempty"`
ExpiresAt *time.Time `json:"expiresAt,omitempty"`
}
type MediaRef struct {
Connector string `json:"connector"`
Auth string `json:"auth,omitempty"`
Path string `json:"path,omitempty"`
Query []QueryParam `json:"query,omitempty"`
AssetID string `json:"assetId,omitempty"`
}

98
db/pluginsystem/installed.go Normal file
View File

@@ -0,0 +1,98 @@
package pluginsystem
import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
)
// LoadInstalledPlugin resolves one plugin from the installed_plugins cache. If
// the cache record is missing or stale, it falls back to the local plugin
// directory so newly copied bundles can still be discovered.
func LoadInstalledPlugin(app core.App, dir string, pluginID string) (LocalPlugin, error) {
if pluginID == "" {
return LocalPlugin{}, fmt.Errorf("plugin id is required")
}
record, _ := app.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": pluginID},
)
if record != nil {
plugin, err := localPluginFromRecord(record)
if err == nil {
return plugin, nil
}
}
if dir == "" {
dir = PluginDir()
}
plugins, err := LoadLocalPlugins(dir)
if err != nil {
return LocalPlugin{}, err
}
for _, plugin := range plugins {
if plugin.Manifest.ID == pluginID {
return plugin, nil
}
}
return LocalPlugin{}, fmt.Errorf("unknown plugin")
}
// LoadInstalledPlugins returns the cached installed plugin manifests used by
// request hot paths, with disk discovery as a bootstrap fallback.
func LoadInstalledPlugins(app core.App, dir string) ([]LocalPlugin, error) {
records, err := app.FindRecordsByFilter("installed_plugins", "", "", -1, 0)
if err != nil {
return nil, err
}
plugins := make([]LocalPlugin, 0, len(records))
for _, record := range records {
plugin, err := localPluginFromRecord(record)
if err != nil {
continue
}
plugins = append(plugins, plugin)
}
if len(plugins) > 0 {
return plugins, nil
}
if dir == "" {
dir = PluginDir()
}
return LoadLocalPlugins(dir)
}
func localPluginFromRecord(record *core.Record) (LocalPlugin, error) {
var manifest Manifest
if err := record.UnmarshalJSONField("manifest", &manifest); err != nil {
return LocalPlugin{}, err
}
if err := ValidateManifest(manifest); err != nil {
return LocalPlugin{}, err
}
dir := strings.TrimSpace(record.GetString("path"))
if dir == "" {
return LocalPlugin{}, fmt.Errorf("installed plugin path is empty")
}
entrypoint := filepath.Clean(manifest.Runtime.Entrypoint)
if filepath.IsAbs(entrypoint) || entrypoint == ".." || strings.HasPrefix(entrypoint, ".."+string(filepath.Separator)) {
return LocalPlugin{}, fmt.Errorf("runtime entrypoint must be relative to plugin directory")
}
wasmPath := filepath.Join(dir, entrypoint)
if _, err := os.Stat(wasmPath); err != nil {
return LocalPlugin{}, fmt.Errorf("runtime entrypoint: %w", err)
}
return LocalPlugin{
Manifest: manifest,
Dir: dir,
WASMPath: wasmPath,
}, nil
}

70
db/pluginsystem/json.go Normal file
View File

@@ -0,0 +1,70 @@
package pluginsystem
import (
"encoding/json"
"github.com/pocketbase/pocketbase/core"
)
// JSONMapFromRecord reads a PocketBase JSON field into a map. Invalid, empty,
// or null values are treated as an empty object because plugin config/state/auth
// fields should be tolerant of partially edited records.
func JSONMapFromRecord(record *core.Record, field string) map[string]any {
if record == nil {
return map[string]any{}
}
value := record.GetString(field)
if value == "" {
return map[string]any{}
}
var result map[string]any
if err := json.Unmarshal([]byte(value), &result); err != nil || result == nil {
return map[string]any{}
}
return result
}
// DeepMergeConfig recursively overlays src onto dst and clones JSON-like values
// so caller-owned config maps cannot be mutated through shared references.
func DeepMergeConfig(dst map[string]any, src map[string]any) {
DeepMergeConfigWithReplaceKeys(dst, src, nil)
}
// DeepMergeConfigWithReplaceKeys behaves like DeepMergeConfig, but map values
// whose key is listed in replaceKeys replace the destination map instead of
// being recursively merged.
func DeepMergeConfigWithReplaceKeys(dst map[string]any, src map[string]any, replaceKeys map[string]bool) {
for key, value := range src {
srcMap, srcIsMap := value.(map[string]any)
dstMap, dstIsMap := dst[key].(map[string]any)
if srcIsMap && dstIsMap {
if replaceKeys[key] {
dst[key] = CloneJSONMap(srcMap)
continue
}
DeepMergeConfigWithReplaceKeys(dstMap, srcMap, replaceKeys)
continue
}
dst[key] = CloneJSONValue(value)
}
}
func CloneJSONMap(values map[string]any) map[string]any {
cloned := make(map[string]any, len(values))
for key, value := range values {
cloned[key] = CloneJSONValue(value)
}
return cloned
}
func CloneJSONValue(value any) any {
data, err := json.Marshal(value)
if err != nil {
return value
}
var cloned any
if err := json.Unmarshal(data, &cloned); err != nil {
return value
}
return cloned
}

81
db/pluginsystem/json_test.go Normal file
View File

@@ -0,0 +1,81 @@
package pluginsystem
import "testing"
func TestMergePluginConfigEmptyCategoryMappingOverridesDefaultMap(t *testing.T) {
dst := map[string]any{
"host": map[string]any{
"categoryMapping": map[string]any{
"hike": "hiking",
"bike": "biking",
},
"privacy": "public",
},
}
src := map[string]any{
"host": map[string]any{
"categoryMapping": map[string]any{},
},
}
MergePluginConfig(dst, src)
host := dst["host"].(map[string]any)
mapping := host["categoryMapping"].(map[string]any)
if len(mapping) != 0 {
t.Fatalf("expected empty category mapping override, got %#v", mapping)
}
if host["privacy"] != "public" {
t.Fatalf("expected sibling defaults to remain, got %#v", host)
}
}
func TestMergePluginConfigCategoryMappingReplacesDefaultMap(t *testing.T) {
dst := map[string]any{
"host": map[string]any{
"categoryMapping": map[string]any{
"hike": "hiking",
"bike": "biking",
},
},
}
src := map[string]any{
"host": map[string]any{
"categoryMapping": map[string]any{
"hike": "custom",
},
},
}
MergePluginConfig(dst, src)
mapping := dst["host"].(map[string]any)["categoryMapping"].(map[string]any)
if len(mapping) != 1 || mapping["hike"] != "custom" {
t.Fatalf("expected category mapping to replace defaults, got %#v", mapping)
}
}
func TestDeepMergeConfigNonEmptyMapStillMergesByDefault(t *testing.T) {
dst := map[string]any{
"host": map[string]any{
"categoryMapping": map[string]any{
"hike": "hiking",
"bike": "biking",
},
},
}
src := map[string]any{
"host": map[string]any{
"categoryMapping": map[string]any{
"hike": "custom",
},
},
}
DeepMergeConfig(dst, src)
mapping := dst["host"].(map[string]any)["categoryMapping"].(map[string]any)
if mapping["hike"] != "custom" || mapping["bike"] != "biking" {
t.Fatalf("expected generic merge to keep sibling defaults, got %#v", mapping)
}
}

419
db/pluginsystem/manager.go Normal file
View File

@@ -0,0 +1,419 @@
package pluginsystem
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"hash/fnv"
"os"
"path/filepath"
"strings"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
)
// Manager coordinates local plugin discovery with the installed_plugins cache.
// It is intentionally small: request hot paths should read cached manifests,
// while list/cron entrypoints refresh the cache from data/plugins first.
type Manager struct {
App core.App
Dir string
}
// PluginInfo is the UI-facing view of an installed plugin. It combines the
// static manifest with runtime availability and embedded icon data.
type PluginInfo struct {
ID string `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
DisplayName string `json:"displayName,omitempty"`
Description string `json:"description,omitempty"`
Icon string `json:"icon,omitempty"`
IconDark string `json:"iconDark,omitempty"`
Version string `json:"version"`
Runtime string `json:"runtime"`
Path string `json:"path"`
Capabilities []string `json:"capabilities"`
Status string `json:"status"`
Error string `json:"error,omitempty"`
Manifest Manifest `json:"manifest"`
}
// NewManager creates a manager for the configured plugin directory. Tests can
// pass a custom dir; production callers use the resolved runtime plugin
// directory.
func NewManager(app core.App, dir string) *Manager {
if dir == "" {
dir = PluginDir()
}
return &Manager{App: app, Dir: dir}
}
// ListLocalPlugins returns installed plugins in the shape consumed by the
// settings UI. It reads from installed_plugins first so listing does not need to
// parse every manifest from disk after the cache has been refreshed.
func (m *Manager) ListLocalPlugins(context.Context) ([]PluginInfo, error) {
plugins, err := LoadInstalledPlugins(m.App, m.Dir)
if err != nil {
return nil, err
}
infos := make([]PluginInfo, 0, len(plugins))
infoByPath := map[string]int{}
for _, plugin := range plugins {
status := "available"
record, _ := m.App.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": plugin.Manifest.ID},
)
if record != nil && record.GetString("status") != "" {
status = record.GetString("status")
}
errorMessage := ""
if record != nil {
errorMessage = record.GetString("error")
}
icon, iconDark := pluginIcons(plugin)
infos = append(infos, PluginInfo{
ID: plugin.Manifest.ID,
Type: plugin.Manifest.Type,
Name: plugin.Manifest.Name,
DisplayName: stringMetadata(plugin.Manifest.Metadata, "displayName"),
Description: plugin.Manifest.Description,
Icon: icon,
IconDark: iconDark,
Version: plugin.Manifest.Version,
Runtime: plugin.Manifest.Runtime.Type,
Path: plugin.Dir,
Capabilities: capabilityNames(plugin.Manifest.Capabilities),
Status: status,
Error: errorMessage,
Manifest: plugin.Manifest,
})
infoByPath[filepath.Clean(plugin.Dir)] = len(infos) - 1
}
_, issues, err := DiscoverLocalPlugins(m.Dir)
if err != nil {
return nil, err
}
for _, issue := range issues {
if index, ok := infoByPath[filepath.Clean(issue.Dir)]; ok {
infos[index].Status = "error"
infos[index].Error = issue.Error
continue
}
infos = append(infos, PluginInfo{
ID: issue.ID,
Type: PluginTypeTrails,
Name: issue.Name,
Path: issue.Dir,
Status: "error",
Error: issue.Error,
Runtime: RuntimeWASM,
Manifest: Manifest{
ID: issue.ID,
Type: PluginTypeTrails,
Name: issue.Name,
Runtime: RuntimeManifest{
Type: RuntimeWASM,
},
},
})
}
return infos, nil
}
// pluginIcons embeds optional light/dark icon files from the plugin bundle as
// data URLs so the frontend does not need direct filesystem access.
func pluginIcons(plugin LocalPlugin) (string, string) {
icons, _ := plugin.Manifest.Metadata["icons"].(map[string]any)
return pluginIcon(plugin.Dir, stringMetadata(icons, "light")), pluginIcon(plugin.Dir, stringMetadata(icons, "dark"))
}
func stringMetadata(values map[string]any, key string) string {
value, _ := values[key].(string)
return value
}
func pluginIcon(pluginDir string, iconPath string) string {
iconPath = strings.TrimSpace(iconPath)
if iconPath == "" {
return ""
}
cleanPath := filepath.Clean(iconPath)
if filepath.IsAbs(cleanPath) || cleanPath == ".." || strings.HasPrefix(cleanPath, ".."+string(filepath.Separator)) {
return ""
}
fullPath := filepath.Join(pluginDir, cleanPath)
data, err := os.ReadFile(fullPath)
if err != nil {
return ""
}
contentType := "image/svg+xml"
switch strings.ToLower(filepath.Ext(fullPath)) {
case ".png":
contentType = "image/png"
case ".jpg", ".jpeg":
contentType = "image/jpeg"
case ".webp":
contentType = "image/webp"
}
return "data:" + contentType + ";base64," + base64.StdEncoding.EncodeToString(data)
}
// SyncInstalledPlugins scans the runtime plugin directory and upserts
// installed_plugins records.
// This keeps the manifest snapshot available even when later code paths should
// avoid repeated disk IO.
func (m *Manager) SyncInstalledPlugins(ctx context.Context) error {
plugins, issues, err := DiscoverLocalPlugins(m.Dir)
if err != nil {
return err
}
collection, err := m.App.FindCollectionByNameOrId("installed_plugins")
if err != nil {
return err
}
activePaths := activePluginPaths(plugins, issues)
if err := m.deleteStaleInstalledPlugins(ctx, activePaths); err != nil {
return err
}
for _, issue := range issues {
if err := ctx.Err(); err != nil {
return err
}
m.App.Logger().Warn("plugin setup error", "plugin", issue.ID, "path", issue.Dir, "error", issue.Error)
if err := m.savePluginIssue(collection, issue); err != nil {
return err
}
}
for _, plugin := range plugins {
if err := ctx.Err(); err != nil {
return err
}
record, err := m.findPluginRecord(collection, plugin)
if err != nil {
return err
}
record.Set("plugin_id", plugin.Manifest.ID)
record.Set("name", plugin.Manifest.Name)
record.Set("type", plugin.Manifest.Type)
record.Set("version", plugin.Manifest.Version)
record.Set("runtime", plugin.Manifest.Runtime.Type)
record.Set("path", plugin.Dir)
record.Set("status", "available")
record.Set("error", "")
manifestJSON, err := marshalManifest(plugin.Manifest)
if err != nil {
return fmt.Errorf("encode installed plugin %s manifest: %w", plugin.Manifest.ID, err)
}
record.Set("manifest", manifestJSON)
record.Set("config", mergeDefaultConfig(defaultConfig(plugin.Manifest), JSONMapFromRecord(record, "config")))
if err := m.App.Save(record); err != nil {
return fmt.Errorf("save installed plugin %s: %w", plugin.Manifest.ID, err)
}
}
return nil
}
func activePluginPaths(plugins []LocalPlugin, issues []LocalPluginIssue) map[string]bool {
paths := make(map[string]bool, len(plugins)+len(issues))
for _, plugin := range plugins {
if plugin.Dir != "" {
paths[filepath.Clean(plugin.Dir)] = true
}
}
for _, issue := range issues {
if issue.Dir != "" {
paths[filepath.Clean(issue.Dir)] = true
}
}
return paths
}
func (m *Manager) deleteStaleInstalledPlugins(ctx context.Context, activePaths map[string]bool) error {
records, err := m.App.FindRecordsByFilter("installed_plugins", "", "", -1, 0)
if err != nil {
return err
}
for _, record := range records {
if err := ctx.Err(); err != nil {
return err
}
path := strings.TrimSpace(record.GetString("path"))
if path != "" && activePaths[filepath.Clean(path)] {
continue
}
if err := m.App.Delete(record); err != nil {
return fmt.Errorf("delete stale installed plugin %s: %w", record.GetString("plugin_id"), err)
}
}
return nil
}
func (m *Manager) findPluginRecord(collection *core.Collection, plugin LocalPlugin) (*core.Record, error) {
recordByID, _ := m.App.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": plugin.Manifest.ID},
)
var recordByPath *core.Record
if plugin.Dir != "" {
recordByPath, _ = m.App.FindFirstRecordByFilter(
"installed_plugins",
"path={:path}",
dbx.Params{"path": plugin.Dir},
)
}
if recordByID != nil && recordByPath != nil && recordByID.Id != recordByPath.Id {
if err := m.App.Delete(recordByPath); err != nil {
return nil, fmt.Errorf("delete superseded installed plugin %s: %w", recordByPath.GetString("plugin_id"), err)
}
}
if recordByID != nil {
return recordByID, nil
}
if recordByPath != nil {
return recordByPath, nil
}
return core.NewRecord(collection), nil
}
func (m *Manager) savePluginIssue(collection *core.Collection, issue LocalPluginIssue) error {
recordID := pluginIssueRecordID(issue)
record, _ := m.App.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": recordID},
)
if record == nil && issue.Dir != "" {
record, _ = m.App.FindFirstRecordByFilter(
"installed_plugins",
"path={:path}",
dbx.Params{"path": issue.Dir},
)
}
if record == nil {
record = core.NewRecord(collection)
record.Set("plugin_id", recordID)
}
record.Set("name", issue.Name)
record.Set("type", PluginTypeTrails)
record.Set("version", "unknown")
record.Set("runtime", RuntimeWASM)
record.Set("path", issue.Dir)
record.Set("manifest", map[string]any{
"id": record.GetString("plugin_id"),
"type": PluginTypeTrails,
"name": issue.Name,
})
record.Set("status", "error")
record.Set("error", issue.Error)
if err := m.App.Save(record); err != nil {
return fmt.Errorf("save plugin setup error %s: %w", issue.ID, err)
}
return nil
}
func pluginIssueRecordID(issue LocalPluginIssue) string {
originalID := strings.TrimSpace(issue.ID)
id := strings.ToLower(originalID)
var builder strings.Builder
for _, r := range id {
switch {
case r >= 'a' && r <= 'z':
builder.WriteRune(r)
case r >= '0' && r <= '9':
builder.WriteRune(r)
case r == '_' || r == '-':
builder.WriteRune(r)
default:
builder.WriteRune('-')
}
}
result := strings.Trim(builder.String(), "-_")
if result == "" {
result = "plugin-setup-error"
}
if originalID != result || !pluginIDPattern.MatchString(result) {
result = strings.Trim(result, "-_")
if result == "" {
result = "plugin-setup-error"
}
result = result + "-" + pluginIssueHash(issue)
}
if len(result) > 128 {
hash := pluginIssueHash(issue)
prefixLength := 128 - len(hash) - 1
result = strings.Trim(result[:prefixLength], "-_") + "-" + hash
}
if pluginIDPattern.MatchString(result) {
return result
}
return "plugin-setup-error-" + pluginIssueHash(issue)
}
func pluginIssueHash(issue LocalPluginIssue) string {
hash := fnv.New32a()
_, _ = hash.Write([]byte(issue.Dir))
_, _ = hash.Write([]byte{0})
_, _ = hash.Write([]byte(issue.ID))
return fmt.Sprintf("%08x", hash.Sum32())
}
func marshalManifest(manifest Manifest) (map[string]any, error) {
data, err := json.Marshal(manifest)
if err != nil {
return nil, err
}
var result map[string]any
if err := json.Unmarshal(data, &result); err != nil {
return nil, err
}
return result, nil
}
func defaultConfig(manifest Manifest) map[string]any {
hostConfig, _ := CloneJSONValue(manifest.HostConfig).(map[string]any)
if hostConfig == nil {
hostConfig = map[string]any{}
}
config := map[string]any{
"host": hostConfig,
}
pluginConfig := map[string]any{}
for _, field := range manifest.ConfigSchema {
if field.Key == "" || field.Default == nil {
continue
}
pluginConfig[field.Key] = CloneJSONValue(field.Default)
}
config["plugin"] = pluginConfig
return config
}
func mergeDefaultConfig(defaults map[string]any, current map[string]any) map[string]any {
if len(defaults) == 0 {
return current
}
merged := CloneJSONMap(defaults)
MergePluginConfig(merged, current)
return merged
}
func MergePluginConfig(dst map[string]any, src map[string]any) {
DeepMergeConfigWithReplaceKeys(dst, src, map[string]bool{
"categoryMapping": true,
})
}
func capabilityNames(capabilities []CapabilityManifest) []string {
names := make([]string, 0, len(capabilities))
for _, capability := range capabilities {
names = append(names, capability.Name+"."+capability.Version)
}
return names
}

21
db/pluginsystem/manager_test.go Normal file
View File

@@ -0,0 +1,21 @@
package pluginsystem
import "testing"
func TestPluginIssueRecordID(t *testing.T) {
valid := pluginIssueRecordID(LocalPluginIssue{ID: "komoot", Dir: "/plugins/komoot"})
if valid != "komoot" {
t.Fatalf("pluginIssueRecordID(valid) = %q, want komoot", valid)
}
first := pluginIssueRecordID(LocalPluginIssue{ID: "@@@", Dir: "/plugins/@@@"})
second := pluginIssueRecordID(LocalPluginIssue{ID: "***", Dir: "/plugins/***"})
if first == second {
t.Fatalf("invalid plugin issue ids collided: %q", first)
}
for _, got := range []string{first, second} {
if !pluginIDPattern.MatchString(got) {
t.Fatalf("pluginIssueRecordID() = %q, not a valid plugin id", got)
}
}
}

305
db/pluginsystem/manifest.go Normal file
View File

@@ -0,0 +1,305 @@
package pluginsystem
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
)
const (
DefaultPluginDir = "/data/plugins"
)
var pluginIDPattern = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]*$`)
var ErrUnsupportedPluginType = errors.New("unsupported plugin type")
type LocalPlugin struct {
Manifest Manifest `json:"manifest"`
Dir string `json:"dir"`
WASMPath string `json:"wasmPath"`
}
// PluginDir resolves the runtime plugin directory. Production containers mount
// plugins at /data/plugins; source checkouts usually stage them at data/plugins
// and may start PocketBase either from the repo root or from db/.
func PluginDir() string {
for _, candidate := range []string{
DefaultPluginDir,
"data/plugins",
filepath.Join("..", "data", "plugins"),
} {
if info, err := os.Stat(candidate); err == nil && info.IsDir() {
return candidate
}
}
return DefaultPluginDir
}
// LoadLocalPlugins reads direct child directories from the plugin directory and
// returns every valid bundle. Invalid direct children are ignored by this
// compatibility helper; callers that need UI-visible errors should use
// DiscoverLocalPlugins.
func LoadLocalPlugins(dir string) ([]LocalPlugin, error) {
plugins, _, err := DiscoverLocalPlugins(dir)
return plugins, err
}
type LocalPluginIssue struct {
ID string
Name string
Dir string
Error string
}
// DiscoverLocalPlugins reads direct child directories from the plugin directory
// and returns valid bundles plus per-directory load issues.
func DiscoverLocalPlugins(dir string) ([]LocalPlugin, []LocalPluginIssue, error) {
if dir == "" {
dir = PluginDir()
}
if _, err := os.Stat(dir); err != nil {
if os.IsNotExist(err) {
return []LocalPlugin{}, nil, nil
}
return nil, nil, err
}
plugins := make([]LocalPlugin, 0)
issues := make([]LocalPluginIssue, 0)
seen := map[string]bool{}
entries, err := os.ReadDir(dir)
if err != nil {
return nil, nil, err
}
for _, entry := range entries {
if !entry.IsDir() {
continue
}
pluginDir := filepath.Join(dir, entry.Name())
plugin, err := LoadLocalPlugin(pluginDir)
if err != nil {
if errors.Is(err, ErrUnsupportedPluginType) {
continue
}
issues = append(issues, LocalPluginIssue{
ID: entry.Name(),
Name: entry.Name(),
Dir: pluginDir,
Error: fmt.Sprintf("%s: %v", entry.Name(), err),
})
continue
}
if seen[plugin.Manifest.ID] {
continue
}
seen[plugin.Manifest.ID] = true
plugins = append(plugins, *plugin)
}
return plugins, issues, nil
}
// LoadLocalPlugin reads one plugin bundle, validates its manifest, and resolves
// the WASM entrypoint relative to the plugin directory.
func LoadLocalPlugin(dir string) (*LocalPlugin, error) {
manifestPath := filepath.Join(dir, "plugin.json")
data, err := os.ReadFile(manifestPath)
if err != nil {
return nil, err
}
var manifest Manifest
if err := json.Unmarshal(data, &manifest); err != nil {
return nil, fmt.Errorf("parse plugin.json: %w", err)
}
if err := ValidateManifest(manifest); err != nil {
return nil, err
}
entrypoint := filepath.Clean(manifest.Runtime.Entrypoint)
if filepath.IsAbs(entrypoint) || strings.HasPrefix(entrypoint, ".."+string(filepath.Separator)) || entrypoint == ".." {
return nil, fmt.Errorf("runtime entrypoint must be relative to plugin directory")
}
wasmPath := filepath.Join(dir, entrypoint)
if _, err := os.Stat(wasmPath); err != nil {
return nil, fmt.Errorf("runtime entrypoint: %w", err)
}
return &LocalPlugin{
Manifest: manifest,
Dir: dir,
WASMPath: wasmPath,
}, nil
}
// ValidateManifest checks the static contract that is trusted by install,
// runtime policy enforcement, auth handling, and the UI.
func ValidateManifest(manifest Manifest) error {
if manifest.ManifestVersion == "" {
return fmt.Errorf("manifestVersion is required")
}
if majorVersion(manifest.ManifestVersion) != majorVersion(ManifestVersion) {
return fmt.Errorf("unsupported manifestVersion %q", manifest.ManifestVersion)
}
if !pluginIDPattern.MatchString(manifest.ID) {
return fmt.Errorf("id must match %s", pluginIDPattern.String())
}
if manifest.Type != PluginTypeTrails {
return fmt.Errorf("%w: type must be %q", ErrUnsupportedPluginType, PluginTypeTrails)
}
if strings.TrimSpace(manifest.Name) == "" {
return fmt.Errorf("name is required")
}
if strings.TrimSpace(manifest.Version) == "" {
return fmt.Errorf("version is required")
}
if manifest.Runtime.Type != RuntimeWASM {
return fmt.Errorf("runtime.type must be %q", RuntimeWASM)
}
if strings.TrimSpace(manifest.Runtime.Entrypoint) == "" {
return fmt.Errorf("runtime.entrypoint is required")
}
if len(manifest.Capabilities) == 0 {
return fmt.Errorf("at least one capability is required")
}
if err := validateCapabilities(manifest.Capabilities); err != nil {
return err
}
if err := validateAuth(manifest.Auth); err != nil {
return err
}
if err := validatePermissions(manifest.Permissions, manifest.Auth); err != nil {
return err
}
return nil
}
func validateCapabilities(capabilities []CapabilityManifest) error {
seen := map[string]bool{}
for _, capability := range capabilities {
if strings.TrimSpace(capability.Name) == "" {
return fmt.Errorf("capability name is required")
}
if strings.TrimSpace(capability.Version) == "" {
return fmt.Errorf("capability %s version is required", capability.Name)
}
if strings.TrimSpace(capability.Export) == "" {
return fmt.Errorf("capability %s export is required", capability.Name)
}
key := capability.Name + "." + capability.Version
if seen[key] {
return fmt.Errorf("duplicate capability %s", key)
}
seen[key] = true
}
return nil
}
func validateAuth(auth AuthManifest) error {
for name, context := range auth.Contexts {
if strings.TrimSpace(name) == "" {
return fmt.Errorf("auth context name is required")
}
if err := ValidateAuthContext(name, context); err != nil {
return err
}
}
return nil
}
func validatePermissions(permissions PermissionManifest, auth AuthManifest) error {
authContexts := map[string]bool{}
for name := range auth.Contexts {
authContexts[name] = true
}
for _, authRef := range permissions.Auth {
if !authContexts[authRef] {
return fmt.Errorf("permission references unknown auth context %q", authRef)
}
}
if err := validateConnectors(permissions.Network.Connectors, authContexts); err != nil {
return err
}
for _, host := range permissions.Network.Redirects.Hosts {
if err := validateHost(host); err != nil {
return err
}
}
if permissions.Network.Redirects.Mode != "" && permissions.Network.Redirects.Mode != "declared_hosts_only" {
return fmt.Errorf("unsupported redirect mode %q", permissions.Network.Redirects.Mode)
}
if permissions.Downloads.MaxBytes < 0 || permissions.Uploads.MaxBytes < 0 {
return fmt.Errorf("maxBytes must not be negative")
}
return nil
}
func validateConnectors(connectors []ConnectorTargetPermission, authContexts map[string]bool) error {
seen := map[string]bool{}
for _, connector := range connectors {
if strings.TrimSpace(connector.Name) == "" {
return fmt.Errorf("connector name is required")
}
if seen[connector.Name] {
return fmt.Errorf("duplicate connector %q", connector.Name)
}
seen[connector.Name] = true
switch connector.Type {
case ConnectorTypePublicAPI:
if strings.TrimSpace(connector.FixedBaseURL) == "" {
return fmt.Errorf("public_api connector %q requires fixedBaseURL", connector.Name)
}
if strings.TrimSpace(connector.ConfigKey) != "" {
return fmt.Errorf("public_api connector %q must not declare configKey", connector.Name)
}
if _, _, err := NormalizeConnectorBase(connector.FixedBaseURL, ""); err != nil {
return fmt.Errorf("connector %q fixedBaseURL: %w", connector.Name, err)
}
case ConnectorTypeConfigured:
if strings.TrimSpace(connector.ConfigKey) == "" {
return fmt.Errorf("configured connector %q requires configKey", connector.Name)
}
if strings.TrimSpace(connector.FixedBaseURL) != "" {
return fmt.Errorf("configured connector %q must not declare fixedBaseURL", connector.Name)
}
default:
return fmt.Errorf("connector %q has unsupported type %q", connector.Name, connector.Type)
}
for _, authRef := range connector.Auth {
if !authContexts[authRef] {
return fmt.Errorf("connector %q references unknown auth context %q", connector.Name, authRef)
}
}
for _, prefix := range connector.AllowedPathPrefixes {
if _, err := CanonicalURLPath(prefix); err != nil {
return fmt.Errorf("connector %q path prefix %q: %w", connector.Name, prefix, err)
}
}
}
return nil
}
func validateHost(host string) error {
host = strings.TrimSpace(host)
if host == "" {
return fmt.Errorf("network host must not be empty")
}
if strings.Contains(host, "://") || strings.Contains(host, "/") {
return fmt.Errorf("network host %q must be a hostname, not a URL", host)
}
return nil
}
func majorVersion(version string) string {
for i, r := range version {
if r == '.' {
return version[:i]
}
}
return version
}

222
db/pluginsystem/manifest_test.go Normal file
View File

@@ -0,0 +1,222 @@
package pluginsystem
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestValidateManifestAcceptsHammerheadShape(t *testing.T) {
manifest := hammerheadManifestForTest()
if err := ValidateManifest(manifest); err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func TestValidateManifestRejectsUnknownAuthPermission(t *testing.T) {
manifest := hammerheadManifestForTest()
manifest.Permissions.Auth = []string{"missing"}
if err := ValidateManifest(manifest); err == nil {
t.Fatal("expected error")
}
}
func TestLoadLocalPluginRequiresRelativeEntrypoint(t *testing.T) {
dir := t.TempDir()
manifest := hammerheadManifestForTest()
manifest.Runtime.Entrypoint = "/tmp/plugin.wasm"
writeManifest(t, dir, manifest)
if _, err := LoadLocalPlugin(dir); err == nil {
t.Fatal("expected error")
}
}
func TestLoadLocalPluginsSkipsMissingPluginDir(t *testing.T) {
plugins, err := LoadLocalPlugins(filepath.Join(t.TempDir(), "missing"))
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(plugins) != 0 {
t.Fatalf("got %d plugins, want 0", len(plugins))
}
}
func TestLoadLocalPluginsFindsDirectChildPlugins(t *testing.T) {
root := t.TempDir()
writePluginDir(t, root, "hammerhead")
writePluginDir(t, root, "komoot")
plugins, err := LoadLocalPlugins(root)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(plugins) != 2 {
t.Fatalf("got %d plugins, want 2", len(plugins))
}
}
func TestDiscoverLocalPluginsReportsMissingManifest(t *testing.T) {
root := t.TempDir()
brokenDir := filepath.Join(root, "komoot")
if err := os.MkdirAll(brokenDir, 0o700); err != nil {
t.Fatalf("mkdir plugin dir: %v", err)
}
plugins, issues, err := DiscoverLocalPlugins(root)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(plugins) != 0 {
t.Fatalf("got %d plugins, want 0", len(plugins))
}
if len(issues) != 1 {
t.Fatalf("got %d issues, want 1", len(issues))
}
if issues[0].ID != "komoot" || issues[0].Name != "komoot" || issues[0].Dir != brokenDir {
t.Fatalf("unexpected issue: %#v", issues[0])
}
if issues[0].Error == "" || !strings.Contains(issues[0].Error, "plugin.json") {
t.Fatalf("expected useful plugin.json error, got %#v", issues[0])
}
}
func TestLoadLocalPluginsIgnoresMissingManifestForCompatibility(t *testing.T) {
root := t.TempDir()
if err := os.MkdirAll(filepath.Join(root, "komoot"), 0o700); err != nil {
t.Fatalf("mkdir plugin dir: %v", err)
}
plugins, err := LoadLocalPlugins(root)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(plugins) != 0 {
t.Fatalf("got %d plugins, want 0", len(plugins))
}
}
func TestLoadLocalPluginsSkipsUnsupportedPluginTypes(t *testing.T) {
root := t.TempDir()
writePluginDir(t, root, "hammerhead")
assetsDir := filepath.Join(root, "immich")
if err := os.MkdirAll(assetsDir, 0o700); err != nil {
t.Fatalf("mkdir plugin dir: %v", err)
}
manifest := hammerheadManifestForTest()
manifest.ID = "immich"
manifest.Name = "Immich"
manifest.Type = "assets"
writeManifest(t, assetsDir, manifest)
if err := os.WriteFile(filepath.Join(assetsDir, "plugin.wasm"), []byte("wasm"), 0o600); err != nil {
t.Fatalf("write wasm: %v", err)
}
plugins, err := LoadLocalPlugins(root)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(plugins) != 1 {
t.Fatalf("got %d plugins, want 1", len(plugins))
}
if plugins[0].Manifest.ID != "hammerhead" {
t.Fatalf("got plugin %q, want hammerhead", plugins[0].Manifest.ID)
}
}
func TestLoadLocalPluginsDoesNotSearchRecursively(t *testing.T) {
root := t.TempDir()
writePluginDir(t, filepath.Join(root, "nested"), "hammerhead")
plugins, err := LoadLocalPlugins(root)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(plugins) != 0 {
t.Fatalf("got %d plugins, want 0", len(plugins))
}
}
func hammerheadManifestForTest() Manifest {
return Manifest{
ManifestVersion: ManifestVersion,
ID: "hammerhead",
Type: PluginTypeTrails,
Name: "Hammerhead",
Version: "0.1.0",
Runtime: RuntimeManifest{
Type: RuntimeWASM,
Entrypoint: "plugin.wasm",
},
Capabilities: []CapabilityManifest{
{Name: "prepare_trail_send", Version: "v1", Export: "prepare_trail_send_v1"},
},
Auth: AuthManifest{
Contexts: map[string]AuthContext{
"provider_session": {
Type: AuthTypeSession,
SecretFields: []string{"email", "password"},
Refresh: &AuthRefresh{
Mode: AuthRefreshModePlugin,
Function: "refresh_session_v1",
},
},
},
},
Permissions: PermissionManifest{
Network: NetworkPermissions{
Connectors: []ConnectorTargetPermission{{
Name: "api",
Type: ConnectorTypePublicAPI,
FixedBaseURL: "https://dashboard.hammerhead.io",
AllowedPathPrefixes: []string{"/v1"},
Auth: []string{"provider_session"},
}},
},
Auth: []string{"provider_session"},
Uploads: UploadPermissions{
MaxBytes: 10 << 20,
ContentTypes: []string{"application/gpx+xml", "application/xml"},
},
},
}
}
func writeManifest(t *testing.T, dir string, manifest Manifest) {
t.Helper()
data := []byte(`{
"manifestVersion": "1.0",
"id": "` + manifest.ID + `",
"type": "` + manifest.Type + `",
"name": "` + manifest.Name + `",
"version": "` + manifest.Version + `",
"runtime": {
"type": "` + manifest.Runtime.Type + `",
"entrypoint": "` + manifest.Runtime.Entrypoint + `"
},
"capabilities": [
{"name": "prepare_trail_send", "version": "v1", "export": "prepare_trail_send_v1"}
]
}`)
if err := os.WriteFile(filepath.Join(dir, "plugin.json"), data, 0o600); err != nil {
t.Fatalf("write manifest: %v", err)
}
}
func writePluginDir(t *testing.T, root string, id string) {
t.Helper()
dir := filepath.Join(root, id)
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatalf("mkdir plugin dir: %v", err)
}
manifest := hammerheadManifestForTest()
manifest.ID = id
manifest.Name = id
writeManifest(t, dir, manifest)
if err := os.WriteFile(filepath.Join(dir, "plugin.wasm"), []byte("wasm"), 0o600); err != nil {
t.Fatalf("write wasm: %v", err)
}
}

335
db/pluginsystem/oauth.go Normal file
View File

@@ -0,0 +1,335 @@
package pluginsystem
import (
"bytes"
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"slices"
"strings"
"time"
"github.com/pocketbase/pocketbase/core"
)
const (
AuthFieldOAuthContext = "oauthContext"
AuthFieldTokenType = "tokenType"
AuthFieldExpiresAt = "expiresAt"
AuthFieldScope = "scope"
)
type OAuthTokenResponse struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token,omitempty"`
TokenType string `json:"token_type,omitempty"`
ExpiresIn int `json:"expires_in,omitempty"`
Scope string `json:"scope,omitempty"`
Raw json.RawMessage `json:"-"`
}
// OAuthContext selects the OAuth auth context declared by a plugin. When the UI
// does not request a specific context, the first context by name is used.
func OAuthContext(plugin LocalPlugin, requested string) (string, AuthContext, error) {
names := make([]string, 0, len(plugin.Manifest.Auth.Contexts))
for name := range plugin.Manifest.Auth.Contexts {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
context := plugin.Manifest.Auth.Contexts[name]
if requested != "" && requested != name {
continue
}
if context.Type == AuthTypeOAuth2 {
return name, context, nil
}
}
return "", AuthContext{}, fmt.Errorf("plugin has no oauth auth context")
}
// ValidateOAuthRedirectURI accepts only the frontend plugin OAuth callback and,
// when ORIGIN is configured, requires the same external origin.
func ValidateOAuthRedirectURI(raw string) error {
redirectURL, err := url.Parse(raw)
if err != nil {
return err
}
if redirectURL.Scheme != "http" && redirectURL.Scheme != "https" {
return fmt.Errorf("redirect uri scheme must be http or https")
}
if redirectURL.Host == "" {
return fmt.Errorf("redirect uri must be absolute")
}
if redirectURL.Path != "/settings/plugins/oauth/callback" {
return fmt.Errorf("redirect uri path is not allowed")
}
if origin := strings.TrimRight(os.Getenv("ORIGIN"), "/"); origin != "" {
originURL, err := url.Parse(origin)
if err != nil {
return err
}
if !strings.EqualFold(redirectURL.Scheme, originURL.Scheme) || !strings.EqualFold(redirectURL.Host, originURL.Host) {
return fmt.Errorf("redirect uri origin does not match ORIGIN")
}
}
return nil
}
func NewOAuthState(size int) string {
return randomURLToken(size)
}
func NewOAuthCodeVerifier(size int) string {
return randomURLToken(size)
}
func PKCEChallenge(verifier string) string {
hash := sha256.Sum256([]byte(verifier))
return base64.RawURLEncoding.EncodeToString(hash[:])
}
// ExchangeOAuthToken performs the host-owned OAuth token exchange or refresh.
// The token endpoint must be allowed by the plugin manifest network policy.
func ExchangeOAuthToken(ctx context.Context, manifest Manifest, authContext AuthContext, auth map[string]any, values map[string]string) (*OAuthTokenResponse, error) {
tokenURL, err := url.Parse(authContext.TokenURL)
if err != nil {
return nil, err
}
if tokenURL.Scheme != "http" && tokenURL.Scheme != "https" {
return nil, fmt.Errorf("oauth token url scheme must be http or https")
}
if !OAuthTokenURLAllowed(manifest, tokenURL) {
return nil, fmt.Errorf("oauth token host %q is not allowed by manifest permissions", tokenURL.Hostname())
}
clientID := StringFromAny(auth["clientId"])
clientSecret := StringFromAny(auth[AuthFieldClientSecret])
if clientID == "" {
return nil, fmt.Errorf("clientId is required")
}
bodyValues := url.Values{}
for key, value := range values {
if value != "" {
bodyValues.Set(key, value)
}
}
bodyValues.Set("client_id", clientID)
if authContext.TokenAuth == "" || authContext.TokenAuth == TokenAuthClientSecretPost {
if clientSecret != "" {
bodyValues.Set("client_secret", clientSecret)
}
}
var body []byte
contentType := "application/x-www-form-urlencoded"
if authContext.TokenRequestFormat == TokenRequestFormatJSON {
jsonBody := map[string]string{}
for key, value := range bodyValues {
if len(value) > 0 {
jsonBody[key] = value[0]
}
}
var err error
body, err = json.Marshal(jsonBody)
if err != nil {
return nil, err
}
contentType = "application/json"
} else {
body = []byte(bodyValues.Encode())
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, tokenURL.String(), bytes.NewReader(body))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", contentType)
req.Header.Set("Accept", "application/json")
if authContext.TokenAuth == TokenAuthClientSecretBasic && clientSecret != "" {
req.SetBasicAuth(clientID, clientSecret)
}
client := &http.Client{
Timeout: 30 * time.Second,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
respBody, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("%s: %s", resp.Status, strings.TrimSpace(string(respBody)))
}
var token OAuthTokenResponse
token.Raw = append([]byte{}, respBody...)
if err := json.Unmarshal(respBody, &token); err != nil {
return nil, err
}
if token.AccessToken == "" {
return nil, fmt.Errorf("oauth token response has no access_token")
}
return &token, nil
}
func OAuthTokenURLAllowed(manifest Manifest, tokenURL *url.URL) bool {
for _, connector := range manifest.Permissions.Network.Connectors {
if connector.Type != ConnectorTypePublicAPI {
continue
}
baseURL, basePath, err := NormalizeConnectorBase(connector.FixedBaseURL, "")
if err != nil {
continue
}
target := ResolvedConnectorTarget{
Name: connector.Name,
Type: connector.Type,
BaseURL: baseURL,
BasePath: basePath,
AllowedPathPrefixes: connector.AllowedPathPrefixes,
}
if err := ValidateConnectorURL(target, tokenURL); err == nil {
return true
}
}
return false
}
// RefreshOAuthToken uses the stored refresh token, persists the refreshed auth
// map, and keeps the plugin instance configured when refresh succeeds.
func RefreshOAuthToken(ctx context.Context, app core.App, plugin LocalPlugin, instance *core.Record, auth map[string]any, contextName string) (map[string]any, error) {
_, authContext, err := OAuthContext(plugin, contextName)
if err != nil {
return auth, err
}
grantType := "refresh_token"
if authContext.Refresh != nil && authContext.Refresh.GrantType != "" {
grantType = authContext.Refresh.GrantType
}
refreshToken := StringFromAny(auth[AuthFieldRefreshToken])
if refreshToken == "" {
return auth, fmt.Errorf("refreshToken is missing")
}
token, err := ExchangeOAuthToken(ctx, plugin.Manifest, authContext, auth, map[string]string{
"grant_type": grantType,
"refresh_token": refreshToken,
})
if err != nil {
return auth, err
}
if token.RefreshToken == "" {
token.RefreshToken = refreshToken
}
StoreOAuthToken(auth, contextName, token)
instance.Set("auth", auth)
instance.Set("status", "configured")
if err := app.Save(instance); err != nil {
return auth, err
}
return auth, nil
}
// StoreOAuthToken normalizes provider token responses into the plugin instance
// auth map used by host injection and future refreshes.
func StoreOAuthToken(auth map[string]any, contextName string, token *OAuthTokenResponse) {
auth[AuthFieldOAuthContext] = contextName
auth[AuthFieldAccessToken] = token.AccessToken
if token.RefreshToken != "" {
auth[AuthFieldRefreshToken] = token.RefreshToken
}
if token.TokenType != "" {
auth[AuthFieldTokenType] = token.TokenType
}
if token.Scope != "" {
auth[AuthFieldScope] = token.Scope
}
if token.ExpiresIn > 0 {
auth[AuthFieldExpiresAt] = time.Now().Add(time.Duration(token.ExpiresIn) * time.Second).UTC().Format(time.RFC3339)
}
}
// ClearOAuthToken removes persisted OAuth token material and transient OAuth
// flow fields from an auth map.
func ClearOAuthToken(auth map[string]any) {
for _, key := range []string{
AuthFieldAccessToken,
AuthFieldRefreshToken,
AuthFieldTokenType,
AuthFieldExpiresAt,
AuthFieldScope,
AuthFieldOAuthState,
AuthFieldOAuthCodeVerifier,
AuthFieldOAuthRedirectURI,
} {
delete(auth, key)
}
}
// PluginInputAuth returns the auth payload visible to plugin exports. OAuth
// token material is intentionally removed because provider requests should go
// through host auth injection instead.
func PluginInputAuth(plugin LocalPlugin, auth map[string]any) map[string]any {
out := map[string]any{}
for key, value := range auth {
out[key] = value
}
for _, context := range plugin.Manifest.Auth.Contexts {
if context.Type == AuthTypeOAuth2 {
for _, key := range PluginInputAuthBlockedFields() {
delete(out, key)
}
}
}
return out
}
// RefreshOAuthAuthIfNeeded refreshes host-managed OAuth before a sync run if no
// access token exists or the current token is close to expiry.
func RefreshOAuthAuthIfNeeded(ctx context.Context, app core.App, plugin LocalPlugin, instance *core.Record, auth map[string]any) (map[string]any, error) {
for name, authContext := range plugin.Manifest.Auth.Contexts {
if authContext.Type != AuthTypeOAuth2 {
continue
}
if StringFromAny(auth[AuthFieldAccessToken]) == "" || OAuthNeedsRefresh(auth) {
return RefreshOAuthToken(ctx, app, plugin, instance, auth, name)
}
}
return auth, nil
}
func OAuthNeedsRefresh(auth map[string]any) bool {
expiresAt := StringFromAny(auth[AuthFieldExpiresAt])
if expiresAt == "" {
return false
}
parsed, err := time.Parse(time.RFC3339, expiresAt)
if err != nil {
return false
}
return time.Until(parsed) < time.Minute
}
func StringFromAny(value any) string {
text, _ := value.(string)
return strings.TrimSpace(text)
}
func randomURLToken(size int) string {
data := make([]byte, size)
if _, err := rand.Read(data); err != nil {
panic(err)
}
return base64.RawURLEncoding.EncodeToString(data)
}

417
db/pluginsystem/policy.go Normal file
View File

@@ -0,0 +1,417 @@
package pluginsystem
import (
"fmt"
"net/url"
"path"
"slices"
"strings"
)
const (
ConnectorTypePublicAPI = "public_api"
ConnectorTypeConfigured = "configured"
TLSModeSystem = "system"
TLSModeCustomCA = "customCA"
)
type RequestPolicyContext struct {
Connectors map[string]ResolvedConnectorTarget
HostAuth map[string]any
}
func (p RequestPolicyContext) WithHostAuth(auth map[string]any) RequestPolicyContext {
p.HostAuth = auth
return p
}
type ResolvedConnectorTarget struct {
Name string
Type string
BaseURL string
BasePath string
AllowPrivate bool
TLS ConnectorTLSConfig
StorageOrigins map[string]ResolvedConnectorOrigin
AllowedPathPrefixes []string
Auth []string
SupportsMediaAuth bool
SupportsStorageRedirects bool
SupportsCustomTLS bool
}
type ConnectorTLSConfig struct {
Mode string
CABundle []byte
}
type ResolvedConnectorOrigin struct {
Name string
BaseURL string
BasePath string
AllowPrivate bool
TLS ConnectorTLSConfig
}
type ResolvedRequestTarget struct {
URL *url.URL
Connector ResolvedConnectorTarget
}
// ValidateHostRequestSpec checks the static manifest policy before the host
// performs any plugin-controlled HTTP request. Provider traffic must use a
// connector target; plugins no longer hand the host absolute API URLs.
func ValidateHostRequestSpec(manifest Manifest, spec HostRequestSpec, policy RequestPolicyContext) error {
_, err := ValidateAndResolveHostRequestSpec(manifest, spec, policy)
return err
}
func ValidateAndResolveHostRequestSpec(manifest Manifest, spec HostRequestSpec, policy RequestPolicyContext) (*ResolvedRequestTarget, error) {
if strings.TrimSpace(spec.Method) == "" {
return nil, fmt.Errorf("method is required")
}
resolved, err := ResolveRequestTarget(manifest, spec.Target, policy)
if err != nil {
return nil, err
}
if spec.Auth != "" {
if err := ValidateAuthReference(manifest, spec.Auth); err != nil {
return nil, err
}
if len(resolved.Connector.Auth) > 0 && !slices.Contains(resolved.Connector.Auth, spec.Auth) {
return nil, fmt.Errorf("auth context %q is not permitted for connector %q", spec.Auth, resolved.Connector.Name)
}
}
if err := validateExpectedResponse(spec.Expect, manifest.Permissions.Downloads); err != nil {
return nil, err
}
return resolved, nil
}
func ValidateAuthReference(manifest Manifest, auth string) error {
if _, ok := manifest.Auth.Contexts[auth]; !ok {
return fmt.Errorf("auth context %q is not declared", auth)
}
if !slices.Contains(manifest.Permissions.Auth, auth) {
return fmt.Errorf("auth context %q is not permitted", auth)
}
return nil
}
func ResolveRequestTarget(manifest Manifest, target RequestTarget, policy RequestPolicyContext) (*ResolvedRequestTarget, error) {
if target.Type != "connector" {
return nil, fmt.Errorf("request target type must be connector")
}
connector, ok := policy.Connectors[target.Connector]
if !ok {
return nil, fmt.Errorf("connector %q is not configured", target.Connector)
}
manifestConnector, ok := manifestConnector(manifest, target.Connector)
if !ok {
return nil, fmt.Errorf("connector %q is not declared by manifest", target.Connector)
}
connector.AllowedPathPrefixes = canonicalConnectorPrefixes(manifestConnector.AllowedPathPrefixes)
connector.Auth = manifestConnector.Auth
connector.SupportsMediaAuth = manifestConnector.SupportsMediaAuth
connector.SupportsStorageRedirects = manifestConnector.SupportsStorageRedirects
connector.SupportsCustomTLS = manifestConnector.SupportsCustomTLS
built, err := BuildConnectorURL(connector, target.Path, target.Query)
if err != nil {
return nil, err
}
if err := ValidateConnectorURL(connector, built); err != nil {
return nil, err
}
return &ResolvedRequestTarget{URL: built, Connector: connector}, nil
}
func manifestConnector(manifest Manifest, name string) (ConnectorTargetPermission, bool) {
for _, connector := range manifest.Permissions.Network.Connectors {
if connector.Name == name {
return connector, true
}
}
return ConnectorTargetPermission{}, false
}
func BuildConnectorURL(connector ResolvedConnectorTarget, relPath string, query []QueryParam) (*url.URL, error) {
base, err := url.Parse(connector.BaseURL)
if err != nil || base.Scheme == "" || base.Host == "" {
return nil, fmt.Errorf("connector %q has invalid baseURL", connector.Name)
}
if base.RawQuery != "" || base.Fragment != "" {
return nil, fmt.Errorf("connector %q baseURL must not include query or fragment", connector.Name)
}
if base.Scheme != "http" && base.Scheme != "https" {
return nil, fmt.Errorf("connector %q scheme must be http or https", connector.Name)
}
base.Path = ""
base.RawPath = ""
cleanBase, err := CanonicalURLPath(connector.BasePath)
if err != nil {
return nil, fmt.Errorf("connector %q basePath: %w", connector.Name, err)
}
cleanRel, err := CanonicalRelativeURLPath(relPath)
if err != nil {
return nil, err
}
fullPath := joinURLPaths(cleanBase, cleanRel)
if strings.HasSuffix(cleanRel, "/") && fullPath != "/" {
fullPath += "/"
}
base.Path = fullPath
encodedQuery := make([]string, 0, len(query))
for _, param := range query {
if hasControl(param.Name) || hasControl(param.Value) {
return nil, fmt.Errorf("query parameters must not contain control characters")
}
if param.Name == "" {
return nil, fmt.Errorf("query parameter name must not be empty")
}
encodedQuery = append(encodedQuery, url.QueryEscape(param.Name)+"="+url.QueryEscape(param.Value))
}
base.RawQuery = strings.Join(encodedQuery, "&")
return base, nil
}
func ValidateConnectorURL(connector ResolvedConnectorTarget, candidate *url.URL) error {
base, err := url.Parse(connector.BaseURL)
if err != nil {
return err
}
if !strings.EqualFold(candidate.Scheme, base.Scheme) {
return fmt.Errorf("connector request scheme escaped scope")
}
if !strings.EqualFold(candidate.Hostname(), base.Hostname()) {
return fmt.Errorf("connector request host escaped scope")
}
if effectivePort(candidate) != effectivePort(base) {
return fmt.Errorf("connector request port escaped scope")
}
candidatePath, err := CanonicalURLPath(candidate.EscapedPath())
if err != nil {
return err
}
basePath, err := CanonicalURLPath(connector.BasePath)
if err != nil {
return err
}
if !pathInPrefix(candidatePath, subtreePrefix(basePath)) {
return fmt.Errorf("connector request escaped base path")
}
prefixes := connector.AllowedPathPrefixes
if len(prefixes) == 0 {
prefixes = []string{"/"}
}
for _, prefix := range canonicalConnectorPrefixes(prefixes) {
fullPrefix := subtreePrefix(joinURLPaths(basePath, prefix))
if pathInPrefix(candidatePath, fullPrefix) {
return nil
}
}
return fmt.Errorf("connector request path is not allowed")
}
func ValidateConnectorRedirect(connector ResolvedConnectorTarget, initial *url.URL, redirected *url.URL) error {
if initial.Scheme == "https" && redirected.Scheme == "http" {
return fmt.Errorf("connector redirect downgrades https to http")
}
return ValidateConnectorURL(connector, redirected)
}
func ValidateConnectorStorageRedirect(connector ResolvedConnectorTarget, initial *url.URL, redirected *url.URL) error {
_, err := ConnectorStorageRedirectOrigin(connector, initial, redirected)
return err
}
func ConnectorStorageRedirectOrigin(connector ResolvedConnectorTarget, initial *url.URL, redirected *url.URL) (ResolvedConnectorOrigin, error) {
if !connector.SupportsStorageRedirects {
return ResolvedConnectorOrigin{}, fmt.Errorf("connector storage redirects are not supported")
}
if initial.Scheme == "https" && redirected.Scheme == "http" {
return ResolvedConnectorOrigin{}, fmt.Errorf("connector storage redirect downgrades https to http")
}
for _, origin := range connector.StorageOrigins {
target := ResolvedConnectorTarget{
Name: origin.Name,
BaseURL: origin.BaseURL,
BasePath: origin.BasePath,
AllowPrivate: origin.AllowPrivate,
TLS: origin.TLS,
AllowedPathPrefixes: []string{"/"},
}
if err := ValidateConnectorURL(target, redirected); err == nil {
return origin, nil
}
}
return ResolvedConnectorOrigin{}, fmt.Errorf("connector storage redirect target is not allowed")
}
func NormalizeConnectorBase(rawURL string, extraBasePath string) (string, string, error) {
parsed, err := url.Parse(strings.TrimSpace(rawURL))
if err != nil || parsed.Scheme == "" || parsed.Host == "" {
return "", "", fmt.Errorf("connector baseURL is invalid")
}
if parsed.Scheme != "http" && parsed.Scheme != "https" {
return "", "", fmt.Errorf("connector baseURL scheme must be http or https")
}
if parsed.RawQuery != "" || parsed.Fragment != "" || parsed.User != nil {
return "", "", fmt.Errorf("connector baseURL must not include credentials, query, or fragment")
}
basePath := parsed.EscapedPath()
if extraBasePath != "" {
basePath = joinURLPaths(basePath, extraBasePath)
}
cleanPath, err := CanonicalURLPath(basePath)
if err != nil {
return "", "", err
}
parsed.Path = ""
parsed.RawPath = ""
return parsed.String(), cleanPath, nil
}
func CanonicalRelativeURLPath(rawPath string) (string, error) {
if strings.TrimSpace(rawPath) == "" {
return "/", nil
}
if strings.HasPrefix(rawPath, "http://") || strings.HasPrefix(rawPath, "https://") || strings.HasPrefix(rawPath, "//") {
return "", fmt.Errorf("connector path must be relative")
}
cleaned, err := CanonicalURLPath("/" + strings.TrimLeft(rawPath, "/"))
if err != nil {
return "", err
}
if strings.HasSuffix(rawPath, "/") && cleaned != "/" {
cleaned += "/"
}
return cleaned, nil
}
func CanonicalURLPath(rawPath string) (string, error) {
if rawPath == "" {
rawPath = "/"
}
if hasControl(rawPath) {
return "", fmt.Errorf("path must not contain control characters")
}
lower := strings.ToLower(rawPath)
if strings.Contains(lower, "%2f") || strings.Contains(lower, "%5c") {
return "", fmt.Errorf("encoded path separators are not allowed")
}
decoded, err := url.PathUnescape(rawPath)
if err != nil {
return "", fmt.Errorf("path has invalid escapes")
}
if strings.Contains(decoded, "\\") {
return "", fmt.Errorf("backslash is not allowed in URL paths")
}
if hasDangerousSecondEscape(decoded) {
return "", fmt.Errorf("ambiguous encoded path is not allowed")
}
cleaned := path.Clean("/" + strings.TrimLeft(decoded, "/"))
if cleaned == "." {
cleaned = "/"
}
return cleaned, nil
}
func canonicalConnectorPrefixes(prefixes []string) []string {
if len(prefixes) == 0 {
return nil
}
canonical := make([]string, 0, len(prefixes))
for _, prefix := range prefixes {
cleaned, err := CanonicalURLPath(prefix)
if err == nil {
canonical = append(canonical, cleaned)
}
}
return canonical
}
func joinURLPaths(left string, right string) string {
if left == "" {
left = "/"
}
if right == "" {
right = "/"
}
joined := path.Join(left, right)
if joined == "." {
return "/"
}
if !strings.HasPrefix(joined, "/") {
joined = "/" + joined
}
return joined
}
func subtreePrefix(prefix string) string {
if prefix == "/" {
return "/"
}
return strings.TrimRight(prefix, "/") + "/"
}
func pathInPrefix(candidate string, prefix string) bool {
if prefix == "/" {
return true
}
candidate = subtreePrefix(candidate)
return strings.HasPrefix(candidate, prefix)
}
func effectivePort(u *url.URL) string {
if port := u.Port(); port != "" {
return port
}
switch u.Scheme {
case "http":
return "80"
case "https":
return "443"
default:
return ""
}
}
func hasControl(value string) bool {
for _, r := range value {
if r < 0x20 || r == 0x7f {
return true
}
}
return false
}
func hasDangerousSecondEscape(value string) bool {
lower := strings.ToLower(value)
for _, marker := range []string{"%2f", "%5c", "%2e"} {
if strings.Contains(lower, marker) {
return true
}
}
return false
}
// validateExpectedResponse lets a plugin request stricter response checks for a
// specific call while preventing it from exceeding manifest download limits.
func validateExpectedResponse(expect ResponseExpect, permissions DownloadPermissions) error {
if expect.MaxBytes < 0 {
return fmt.Errorf("expect.maxBytes must not be negative")
}
if permissions.MaxBytes > 0 && expect.MaxBytes > permissions.MaxBytes {
return fmt.Errorf("expect.maxBytes exceeds manifest download limit")
}
for _, contentType := range expect.ContentTypes {
if len(permissions.ContentTypes) > 0 && !slices.Contains(permissions.ContentTypes, contentType) {
return fmt.Errorf("content type %q is not allowed by manifest permissions", contentType)
}
}
return nil
}

169
db/pluginsystem/policy_test.go Normal file
View File

@@ -0,0 +1,169 @@
package pluginsystem
import (
"net/url"
"testing"
)
func TestValidateHostRequestSpecAcceptsConnectorAndAuthReference(t *testing.T) {
manifest := hammerheadManifestForTest()
spec := HostRequestSpec{
Method: "POST",
Target: RequestTarget{
Type: "connector",
Connector: "api",
Path: "/v1/users/123/routes/import/file",
},
Auth: "provider_session",
Expect: ResponseExpect{
ContentTypes: []string{"application/json"},
MaxBytes: 1024,
},
}
manifest.Permissions.Downloads.ContentTypes = append(manifest.Permissions.Downloads.ContentTypes, "application/json")
manifest.Permissions.Downloads.MaxBytes = 2048
if err := ValidateHostRequestSpec(manifest, spec, testPolicy()); err != nil {
t.Fatalf("unexpected error: %v", err)
}
}
func TestValidateHostRequestSpecRejectsUnknownConnector(t *testing.T) {
manifest := hammerheadManifestForTest()
spec := HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "evil", Path: "/v1"},
}
if err := ValidateHostRequestSpec(manifest, spec, testPolicy()); err == nil {
t.Fatal("expected error")
}
}
func TestValidateHostRequestSpecRejectsPathScopeEscape(t *testing.T) {
manifest := hammerheadManifestForTest()
spec := HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1-evil"},
}
if err := ValidateHostRequestSpec(manifest, spec, testPolicy()); err == nil {
t.Fatal("expected error")
}
}
func TestValidateHostRequestSpecRejectsLimitExpansion(t *testing.T) {
manifest := hammerheadManifestForTest()
manifest.Permissions.Downloads.MaxBytes = 100
spec := HostRequestSpec{
Method: "GET",
Target: RequestTarget{Type: "connector", Connector: "api", Path: "/v1/users"},
Expect: ResponseExpect{MaxBytes: 101},
}
if err := ValidateHostRequestSpec(manifest, spec, testPolicy()); err == nil {
t.Fatal("expected error")
}
}
func TestBuildConnectorURLPreservesBasePathAndQueryOrder(t *testing.T) {
target := ResolvedConnectorTarget{
Name: "immich",
BaseURL: "https://photos.example.test:8443",
BasePath: "/immich",
AllowedPathPrefixes: []string{"/api"},
}
u, err := BuildConnectorURL(target, "/api/assets/1/original", []QueryParam{
{Name: "z", Value: "last"},
{Name: "key", Value: "a"},
{Name: "key", Value: "b"},
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if u.String() != "https://photos.example.test:8443/immich/api/assets/1/original?z=last&key=a&key=b" {
t.Fatalf("unexpected url: %s", u.String())
}
if err := ValidateConnectorURL(target, u); err != nil {
t.Fatalf("unexpected scope error: %v", err)
}
}
func TestBuildConnectorURLPreservesTrailingSlash(t *testing.T) {
target := ResolvedConnectorTarget{
Name: "komoot",
BaseURL: "https://api.komoot.de",
BasePath: "/",
AllowedPathPrefixes: []string{"/v006"},
}
u, err := BuildConnectorURL(target, "/v006/account/email/user%40example.test/", nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if u.String() != "https://api.komoot.de/v006/account/email/user@example.test/" {
t.Fatalf("unexpected url: %s", u.String())
}
}
func TestConnectorPathNormalizationRejectsAmbiguousEscapes(t *testing.T) {
for _, candidate := range []string{"/api%2fadmin", "/api/%252e%252e/admin", "/api/../admin"} {
t.Run(candidate, func(t *testing.T) {
target := ResolvedConnectorTarget{
Name: "api",
BaseURL: "https://example.test",
BasePath: "/",
AllowedPathPrefixes: []string{"/api"},
}
u, err := BuildConnectorURL(target, candidate, nil)
if err == nil {
err = ValidateConnectorURL(target, u)
}
if err == nil {
t.Fatal("expected scope error")
}
})
}
}
func TestConnectorStorageRedirectOriginReturnsMatchedOriginPolicy(t *testing.T) {
connector := ResolvedConnectorTarget{
Name: "immich",
BaseURL: "https://photos.example.test",
BasePath: "/immich",
SupportsStorageRedirects: true,
StorageOrigins: map[string]ResolvedConnectorOrigin{
"minio": {
Name: "minio",
BaseURL: "https://storage.example.test:9443",
BasePath: "/assets",
AllowPrivate: true,
TLS: ConnectorTLSConfig{Mode: TLSModeCustomCA, CABundle: []byte("ca")},
},
},
}
initial, _ := BuildConnectorURL(connector, "/api/assets/1/original", nil)
redirected, _ := url.Parse("https://storage.example.test:9443/assets/bucket/photo.jpg")
origin, err := ConnectorStorageRedirectOrigin(connector, initial, redirected)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if origin.Name != "minio" || !origin.AllowPrivate || origin.TLS.Mode != TLSModeCustomCA {
t.Fatalf("unexpected origin policy: %#v", origin)
}
}
func testPolicy() RequestPolicyContext {
return RequestPolicyContext{
Connectors: map[string]ResolvedConnectorTarget{
"api": {
Name: "api",
Type: ConnectorTypePublicAPI,
BaseURL: "https://dashboard.hammerhead.io",
BasePath: "/",
AllowedPathPrefixes: []string{"/v1"},
Auth: []string{"provider_session"},
},
},
}
}

212
db/pluginsystem/protocol.go Normal file
View File

@@ -0,0 +1,212 @@
package pluginsystem
const (
ManifestVersion = "1.0"
RuntimeWASM = "wasm"
PluginTypeTrails = "trails"
AuthTypeOAuth2 = "oauth2"
AuthTypeAPIKey = "api_key"
AuthTypeBearer = "bearer"
AuthTypeSession = "session"
AuthRefreshModeHost = "host"
AuthRefreshModePlugin = "plugin"
AuthPlacementQuery = "query"
AuthHeaderAuthorization = "Authorization"
AuthSchemeBearer = "Bearer"
TokenRequestFormatJSON = "json"
TokenAuthClientSecretPost = "client_secret_post"
TokenAuthClientSecretBasic = "client_secret_basic"
HostRequestBodyTypeJSON = "json"
HostRequestBodyTypeForm = "form"
HostRequestBodyTypeMultipart = "multipart"
MultipartSourceTrail = "trail"
MultipartSourceTrailGPX = "trail.gpx"
MultipartTrailFilename = "trail.gpx"
)
type Manifest struct {
ManifestVersion string `json:"manifestVersion"`
ID string `json:"id"`
Type string `json:"type"`
Name string `json:"name"`
Description string `json:"description,omitempty"`
Version string `json:"version"`
Runtime RuntimeManifest `json:"runtime"`
Capabilities []CapabilityManifest `json:"capabilities"`
Auth AuthManifest `json:"auth,omitempty"`
Permissions PermissionManifest `json:"permissions,omitempty"`
ConfigSchema []ConfigField `json:"configSchema,omitempty"`
HostConfig map[string]any `json:"hostConfig,omitempty"`
Metadata map[string]any `json:"metadata,omitempty"`
}
type RuntimeManifest struct {
Type string `json:"type"`
Entrypoint string `json:"entrypoint"`
}
type CapabilityManifest struct {
Name string `json:"name"`
Version string `json:"version"`
Export string `json:"export"`
RequiredFunctions []string `json:"requiredHostFunctions,omitempty"`
Job string `json:"job,omitempty"`
}
type ConfigField struct {
Key string `json:"key"`
Type string `json:"type"`
Label string `json:"label,omitempty"`
Labels map[string]string `json:"labels,omitempty"`
Description string `json:"description,omitempty"`
Descriptions map[string]string `json:"descriptions,omitempty"`
Options []ConfigFieldOption `json:"options,omitempty"`
Default any `json:"default,omitempty"`
Required bool `json:"required,omitempty"`
Hidden bool `json:"hidden,omitempty"`
}
type ConfigFieldOption struct {
Value string `json:"value"`
Label string `json:"label,omitempty"`
Labels map[string]string `json:"labels,omitempty"`
}
type AuthManifest struct {
Contexts map[string]AuthContext `json:"contexts,omitempty"`
}
type AuthContext struct {
Type string `json:"type"`
Fields []string `json:"fields,omitempty"`
AuthorizationURL string `json:"authorizationUrl,omitempty"`
TokenURL string `json:"tokenUrl,omitempty"`
Scopes []string `json:"scopes,omitempty"`
ScopeSeparator string `json:"scopeSeparator,omitempty"`
PKCE bool `json:"pkce,omitempty"`
TokenRequestFormat string `json:"tokenRequestFormat,omitempty"`
TokenAuth string `json:"tokenAuth,omitempty"`
AuthorizationParams map[string]string `json:"authorizationParams,omitempty"`
Refresh *AuthRefresh `json:"refresh,omitempty"`
Placement string `json:"placement,omitempty"`
Name string `json:"name,omitempty"`
SecretField string `json:"secretField,omitempty"`
SecretFields []string `json:"secretFields,omitempty"`
}
type AuthRefresh struct {
Mode string `json:"mode"`
GrantType string `json:"grantType,omitempty"`
Function string `json:"function,omitempty"`
}
type PermissionManifest struct {
Network NetworkPermissions `json:"network,omitempty"`
Auth []string `json:"auth,omitempty"`
Downloads DownloadPermissions `json:"downloads,omitempty"`
Uploads UploadPermissions `json:"uploads,omitempty"`
}
type NetworkPermissions struct {
Connectors []ConnectorTargetPermission `json:"connectors,omitempty"`
Redirects RedirectPermissions `json:"redirects,omitempty"`
}
type ConnectorTargetPermission struct {
Name string `json:"name"`
Type string `json:"type"`
FixedBaseURL string `json:"fixedBaseURL,omitempty"`
ConfigKey string `json:"configKey,omitempty"`
AllowedPathPrefixes []string `json:"allowedPathPrefixes,omitempty"`
Auth []string `json:"auth,omitempty"`
SupportsMediaAuth bool `json:"supportsMediaAuth,omitempty"`
SupportsStorageRedirects bool `json:"supportsStorageRedirects,omitempty"`
SupportsCustomTLS bool `json:"supportsCustomTLS,omitempty"`
}
type RedirectPermissions struct {
Mode string `json:"mode,omitempty"`
Hosts []string `json:"hosts,omitempty"`
}
type DownloadPermissions struct {
MaxBytes int64 `json:"maxBytes,omitempty"`
ContentTypes []string `json:"contentTypes,omitempty"`
}
type UploadPermissions struct {
MaxBytes int64 `json:"maxBytes,omitempty"`
ContentTypes []string `json:"contentTypes,omitempty"`
}
type HostRequestSpec struct {
Method string `json:"method"`
Target RequestTarget `json:"target"`
Auth string `json:"auth,omitempty"`
Headers map[string]string `json:"headers,omitempty"`
Body *HostRequestBody `json:"body,omitempty"`
Expect ResponseExpect `json:"expect,omitempty"`
FollowRedirects *bool `json:"followRedirects,omitempty"`
}
type RequestTarget struct {
Type string `json:"type"`
Connector string `json:"connector,omitempty"`
Path string `json:"path,omitempty"`
Query []QueryParam `json:"query,omitempty"`
}
type QueryParam struct {
Name string `json:"name"`
Value string `json:"value"`
}
type HostRequestBody struct {
Type string `json:"type"`
JSON any `json:"json,omitempty"`
Form []FormField `json:"form,omitempty"`
Parts []MultipartPart `json:"parts,omitempty"`
}
type FormField struct {
Name string `json:"name"`
Value string `json:"value"`
}
type MultipartPart struct {
Name string `json:"name"`
Source string `json:"source,omitempty"`
Filename string `json:"filename,omitempty"`
ContentType string `json:"contentType,omitempty"`
JSON any `json:"json,omitempty"`
}
type ResponseExpect struct {
ContentTypes []string `json:"contentTypes,omitempty"`
MaxBytes int64 `json:"maxBytes,omitempty"`
}
type TrackTransferPlan struct {
Format string `json:"format"`
Transfer HostRequestSpec `json:"transfer"`
}
type TrailSendPlan struct {
Request HostRequestSpec `json:"request"`
}
type PluginError struct {
Code string `json:"code"`
Message string `json:"message,omitempty"`
RetryAfterSeconds *int `json:"retryAfterSeconds,omitempty"`
}
type HostLogEntry struct {
Level string `json:"level"`
Message string `json:"message"`
}

64
db/pluginsystem/runtime.go Normal file
View File

@@ -0,0 +1,64 @@
package pluginsystem
import (
"context"
"errors"
"fmt"
)
var ErrRuntimeUnavailable = errors.New("plugin runtime is not available")
type Runtime interface {
Call(ctx context.Context, plugin LocalPlugin, export string, input []byte, policy RequestPolicyContext) ([]byte, error)
OpenSession(ctx context.Context, plugin LocalPlugin, policy RequestPolicyContext) (RuntimeSession, error)
}
type RuntimeSession interface {
Call(ctx context.Context, export string, input []byte) ([]byte, error)
Close(ctx context.Context) error
}
type RuntimeRegistry struct {
wasm Runtime
}
// NewRuntimeRegistry wires available runtime implementations behind the common
// Runtime interface.
func NewRuntimeRegistry() *RuntimeRegistry {
return &RuntimeRegistry{
wasm: NewWorkerRuntime(),
}
}
// RuntimeFor selects the runtime declared by a plugin manifest.
func (r *RuntimeRegistry) RuntimeFor(plugin LocalPlugin) (Runtime, error) {
switch plugin.Manifest.Runtime.Type {
case RuntimeWASM:
return r.wasm, nil
default:
return nil, ErrRuntimeUnavailable
}
}
type UnavailableRuntime struct{}
func (UnavailableRuntime) Call(context.Context, LocalPlugin, string, []byte, RequestPolicyContext) ([]byte, error) {
return nil, ErrRuntimeUnavailable
}
func (UnavailableRuntime) OpenSession(context.Context, LocalPlugin, RequestPolicyContext) (RuntimeSession, error) {
return nil, ErrRuntimeUnavailable
}
type PluginCallError struct {
PluginID string
Export string
PluginError PluginError
}
func (e PluginCallError) Error() string {
if e.PluginError.Message == "" {
return fmt.Sprintf("call %s.%s: %s", e.PluginID, e.Export, e.PluginError.Code)
}
return fmt.Sprintf("call %s.%s: %s: %s", e.PluginID, e.Export, e.PluginError.Code, e.PluginError.Message)
}

89
db/pluginsystem/status.go Normal file
View File

@@ -0,0 +1,89 @@
package pluginsystem
import (
"errors"
"fmt"
"strings"
"time"
)
// PluginCapabilityError wraps a plugin-reported error returned inside a
// successful export response, so status mapping can treat it like runtime
// PluginCallError failures.
type PluginCapabilityError struct {
Err *PluginError
}
func (e PluginCapabilityError) Error() string {
if e.Err == nil {
return "plugin error"
}
if e.Err.Message == "" {
return fmt.Sprintf("plugin error %s", e.Err.Code)
}
return fmt.Sprintf("plugin error %s: %s", e.Err.Code, e.Err.Message)
}
// InstanceStatusUpdate contains the normalized status fields that are written
// back to plugin_instances after a failed sync.
type InstanceStatusUpdate struct {
Status string
Code string
Message string
RetryNotBefore *time.Time
}
// InstanceStatusForError converts sync/runtime errors into the persisted
// plugin_instances status fields used by the UI and cron backoff logic.
func InstanceStatusForError(err error, now time.Time) InstanceStatusUpdate {
var capabilityErr PluginCapabilityError
var callErr PluginCallError
if errors.As(err, &capabilityErr) && capabilityErr.Err != nil {
return InstanceStatusForPluginError(*capabilityErr.Err, now)
}
if errors.As(err, &callErr) {
return InstanceStatusForPluginError(callErr.PluginError, now)
}
return InstanceStatusUpdate{
Status: "error",
Code: "provider_unavailable",
Message: err.Error(),
}
}
// InstanceStatusForPluginError maps the stable plugin error codes from the ABI
// to host instance states. retryAfterSeconds wins over default retry windows.
func InstanceStatusForPluginError(pluginErr PluginError, now time.Time) InstanceStatusUpdate {
code := strings.TrimSpace(pluginErr.Code)
if code == "" {
code = "provider_unavailable"
}
message := strings.TrimSpace(pluginErr.Message)
if message == "" {
message = code
}
status := "error"
switch code {
case "auth_failed", "invalid_grant", "unauthorized":
status = "needs_reauth"
case "rate_limited":
status = "rate_limited"
case "provider_unavailable", "temporary_unavailable":
status = "unavailable"
}
update := InstanceStatusUpdate{
Status: status,
Code: code,
Message: message,
}
if pluginErr.RetryAfterSeconds != nil && *pluginErr.RetryAfterSeconds > 0 {
retryNotBefore := now.Add(time.Duration(*pluginErr.RetryAfterSeconds) * time.Second)
update.RetryNotBefore = &retryNotBefore
} else if code == "rate_limited" {
retryNotBefore := now.Add(time.Hour)
update.RetryNotBefore = &retryNotBefore
}
return update
}

59
db/pluginsystem/status_test.go Normal file
View File

@@ -0,0 +1,59 @@
package pluginsystem
import (
"errors"
"testing"
"time"
)
func TestInstanceStatusForPluginCapabilityError(t *testing.T) {
now := time.Date(2026, 5, 31, 12, 0, 0, 0, time.UTC)
retryAfter := 120
update := InstanceStatusForError(PluginCapabilityError{Err: &PluginError{
Code: "rate_limited",
Message: "try later",
RetryAfterSeconds: &retryAfter,
}}, now)
if update.Status != "rate_limited" {
t.Fatalf("expected status rate_limited, got %q", update.Status)
}
if update.Code != "rate_limited" || update.Message != "try later" {
t.Fatalf("unexpected error fields: %#v", update)
}
if update.RetryNotBefore == nil || !update.RetryNotBefore.Equal(now.Add(120*time.Second)) {
t.Fatalf("unexpected retry time: %#v", update.RetryNotBefore)
}
}
func TestInstanceStatusForPluginCallError(t *testing.T) {
update := InstanceStatusForError(PluginCallError{
PluginID: "strava",
Export: "list_activities_v1",
PluginError: PluginError{
Code: "invalid_grant",
},
}, time.Now())
if update.Status != "needs_reauth" {
t.Fatalf("expected status needs_reauth, got %q", update.Status)
}
if update.Code != "invalid_grant" || update.Message != "invalid_grant" {
t.Fatalf("unexpected error fields: %#v", update)
}
if update.RetryNotBefore != nil {
t.Fatalf("did not expect retry time: %#v", update.RetryNotBefore)
}
}
func TestInstanceStatusForGenericError(t *testing.T) {
update := InstanceStatusForError(errors.New("network unavailable"), time.Now())
if update.Status != "error" {
t.Fatalf("expected status error, got %q", update.Status)
}
if update.Code != "provider_unavailable" || update.Message != "network unavailable" {
t.Fatalf("unexpected error fields: %#v", update)
}
}

507
db/pluginsystem/worker.go Normal file
View File

@@ -0,0 +1,507 @@
package pluginsystem
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"log"
"os"
"os/exec"
"runtime"
"strconv"
"strings"
"sync"
"time"
"golang.org/x/sync/semaphore"
)
const (
defaultWorkerExportTimeout = 2 * time.Minute
defaultWorkerSessionTimeout = 15 * time.Minute
defaultWorkerSlotAcquireTimeout = 30 * time.Second
defaultWorkerCapturedStderrBytes = 64 * 1024
)
var (
workerSlotsMu sync.Mutex
workerSlots *semaphore.Weighted
)
type WorkerRuntime struct {
Executable string
}
type RuntimeSessionFatalError struct {
Err error
}
func (e RuntimeSessionFatalError) Error() string {
return e.Err.Error()
}
func (e RuntimeSessionFatalError) Unwrap() error {
return e.Err
}
func IsRuntimeSessionFatalError(err error) bool {
var fatal RuntimeSessionFatalError
return errors.As(err, &fatal)
}
func NewWorkerRuntime() WorkerRuntime {
return WorkerRuntime{}
}
func (r WorkerRuntime) Call(ctx context.Context, plugin LocalPlugin, export string, input []byte, policy RequestPolicyContext) ([]byte, error) {
session, err := r.OpenSession(ctx, plugin, policy)
if err != nil {
return nil, err
}
defer func() {
_ = session.Close(context.Background())
}()
return session.Call(ctx, export, input)
}
func (r WorkerRuntime) OpenSession(ctx context.Context, plugin LocalPlugin, policy RequestPolicyContext) (RuntimeSession, error) {
slot, err := acquireWorkerSlot(ctx)
if err != nil {
return nil, err
}
releaseSlot := true
defer func() {
if releaseSlot {
slot.Release(1)
}
}()
executable := r.Executable
if executable == "" {
if configured := strings.TrimSpace(os.Getenv("WANDERER_PLUGIN_WORKER_BIN")); configured != "" {
executable = configured
} else {
var err error
executable, err = os.Executable()
if err != nil {
return nil, err
}
}
}
cmd := exec.Command(executable, "plugin-worker")
cmd.Env = childEnvWithout("EXTISM_ENABLE_WASI_OUTPUT")
stdin, err := cmd.StdinPipe()
if err != nil {
return nil, err
}
stdout, err := cmd.StdoutPipe()
if err != nil {
return nil, err
}
stderr := &boundedWorkerBuffer{limit: envInt("WANDERER_PLUGIN_WORKER_STDERR_BYTES", defaultWorkerCapturedStderrBytes)}
cmd.Stderr = stderr
if err := cmd.Start(); err != nil {
return nil, err
}
session := &workerRuntimeSession{
plugin: plugin,
policy: policy,
sessionID: newWorkerSessionID(plugin.Manifest.ID),
cmd: cmd,
stdin: stdin,
stdout: stdout,
stderr: stderr,
requestMaxBytes: envInt("WANDERER_PLUGIN_WORKER_REQUEST_BYTES", defaultWorkerRequestMaxBytes),
responseMaxBytes: envInt("WANDERER_PLUGIN_WORKER_RESPONSE_BYTES", defaultWorkerResponseMaxBytes),
exportTimeout: envDuration("WANDERER_PLUGIN_WORKER_EXPORT_TIMEOUT", defaultWorkerExportTimeout),
slot: slot,
}
session.sessionTimer = time.AfterFunc(envDuration("WANDERER_PLUGIN_WORKER_SESSION_TIMEOUT", defaultWorkerSessionTimeout), func() {
session.markFatal("worker session timeout")
session.kill()
})
releaseSlot = false
return session, nil
}
type workerRuntimeSession struct {
plugin LocalPlugin
policy RequestPolicyContext
sessionID string
cmd *exec.Cmd
stdin io.WriteCloser
stdout io.ReadCloser
stderr *boundedWorkerBuffer
requestMaxBytes int
responseMaxBytes int
exportTimeout time.Duration
sessionTimer *time.Timer
slot *semaphore.Weighted
mu sync.Mutex
callMu sync.Mutex
waitMu sync.Mutex
waited bool
waitErr error
closed bool
fatal bool
fatalMsg string
}
func (s *workerRuntimeSession) Call(ctx context.Context, export string, input []byte) ([]byte, error) {
s.callMu.Lock()
defer s.callMu.Unlock()
s.mu.Lock()
if s.closed {
s.mu.Unlock()
return nil, fmt.Errorf("worker session is closed")
}
if s.fatal {
msg := s.fatalMsg
s.mu.Unlock()
return nil, RuntimeSessionFatalError{Err: fmt.Errorf("worker session is invalid: %s", msg)}
}
s.mu.Unlock()
callCtx, cancel := context.WithTimeout(ctx, s.exportTimeout)
defer cancel()
result := make(chan workerCallOutcome, 1)
go func() {
result <- s.call(callCtx, export, input)
}()
select {
case outcome := <-result:
if outcome.err != nil {
return nil, outcome.err
}
return outcome.output, nil
case <-callCtx.Done():
s.markFatal("worker export timeout")
s.kill()
outcome := <-result
if outcome.err != nil && !errors.Is(outcome.err, io.EOF) {
return nil, RuntimeSessionFatalError{Err: fmt.Errorf("worker export timeout: %w", outcome.err)}
}
return nil, RuntimeSessionFatalError{Err: callCtx.Err()}
}
}
type workerCallOutcome struct {
output []byte
err error
}
func (s *workerRuntimeSession) call(ctx context.Context, export string, input []byte) workerCallOutcome {
msg, err := workerMessageWithData(workerMessageCallExport, workerCallExport{
WASMPath: s.plugin.WASMPath,
Export: export,
InputBase64: encodeWorkerBytes(input),
SessionID: s.sessionID,
})
if err != nil {
return workerCallOutcome{err: err}
}
if err := writeWorkerMessage(s.stdin, s.requestMaxBytes, msg); err != nil {
s.markFatal("write worker call_export failed")
s.kill()
return workerCallOutcome{err: RuntimeSessionFatalError{Err: err}}
}
for {
msg, err := readWorkerMessage(s.stdout, s.responseMaxBytes)
if err != nil {
s.markFatal("read worker message failed")
s.kill()
return workerCallOutcome{err: RuntimeSessionFatalError{Err: s.withStderr(err)}}
}
switch msg.Type {
case workerMessageHostHTTPRequest:
if err := s.handleHostHTTPRequest(ctx, msg); err != nil {
s.markFatal("host http rpc failed")
s.kill()
return workerCallOutcome{err: RuntimeSessionFatalError{Err: s.withStderr(err)}}
}
case workerMessageHostLog:
s.handleHostLog(msg)
case workerMessageCallResult:
result, err := workerData[workerCallResult](msg)
if err != nil {
s.markFatal("invalid call_result payload")
s.kill()
return workerCallOutcome{err: RuntimeSessionFatalError{Err: err}}
}
if result.PluginError != nil {
return workerCallOutcome{err: PluginCallError{
PluginID: s.plugin.Manifest.ID,
Export: export,
PluginError: *result.PluginError,
}}
}
output, err := decodeWorkerBytes(result.OutputBase64)
if err != nil {
s.markFatal("invalid call_result output")
s.kill()
return workerCallOutcome{err: RuntimeSessionFatalError{Err: err}}
}
return workerCallOutcome{output: output}
case workerMessageError:
payload, _ := workerData[workerError](msg)
s.markFatal(payload.Message)
s.kill()
if payload.Message == "" {
payload.Message = "worker returned fatal error"
}
return workerCallOutcome{err: RuntimeSessionFatalError{Err: s.withStderr(fmt.Errorf("%s", payload.Message))}}
default:
s.markFatal("unexpected worker message")
s.kill()
return workerCallOutcome{err: RuntimeSessionFatalError{Err: fmt.Errorf("unexpected worker message %q", msg.Type)}}
}
}
}
func (s *workerRuntimeSession) handleHostLog(msg workerMessage) {
entry, err := workerData[workerHostLog](msg)
if err != nil {
log.Printf("plugin log invalid: session %s: %v", s.sessionID, err)
return
}
if entry.SessionID == "" {
entry.SessionID = s.sessionID
}
level, err := normalizeHostLogLevel(entry.Level)
if err != nil {
log.Printf("plugin log invalid: session %s: %v", s.sessionID, err)
return
}
message := sanitizeHostLogMessage(entry.Message)
if message == "" {
log.Printf("plugin log invalid: session %s: log message is required", s.sessionID)
return
}
log.Printf("plugin log [%s]: session %s: %s", level, entry.SessionID, message)
}
func (s *workerRuntimeSession) handleHostHTTPRequest(ctx context.Context, msg workerMessage) error {
request, err := workerData[workerHostHTTPRequest](msg)
if err != nil {
return err
}
requestBytes, err := decodeWorkerBytes(request.RequestBase64)
if err != nil {
return err
}
response := executeHostHTTPRequest(ctx, s.plugin.Manifest, s.policy, requestBytes)
responseBytes, err := json.Marshal(response)
if err != nil {
return err
}
reply, err := workerMessageWithData(workerMessageHostHTTPResponse, workerHostHTTPResponse{
ResponseBase64: encodeWorkerBytes(responseBytes),
})
if err != nil {
return err
}
return writeWorkerMessage(s.stdin, s.responseMaxBytes, reply)
}
func (s *workerRuntimeSession) Close(ctx context.Context) error {
s.mu.Lock()
if s.closed {
s.mu.Unlock()
return nil
}
s.closed = true
fatal := s.fatal
s.mu.Unlock()
if s.sessionTimer != nil {
s.sessionTimer.Stop()
}
if !fatal {
_ = writeWorkerMessage(s.stdin, s.requestMaxBytes, workerMessage{Type: workerMessageShutdown})
}
_ = s.stdin.Close()
wait := make(chan error, 1)
go func() {
wait <- s.wait()
}()
select {
case err := <-wait:
s.slot.Release(1)
if err != nil && !fatal {
return s.withStderr(err)
}
return nil
case <-ctx.Done():
s.kill()
err := <-wait
s.slot.Release(1)
if err != nil {
return s.withStderr(err)
}
return ctx.Err()
}
}
func (s *workerRuntimeSession) wait() error {
s.waitMu.Lock()
defer s.waitMu.Unlock()
if s.waited {
return s.waitErr
}
s.waited = true
s.waitErr = s.cmd.Wait()
return s.waitErr
}
func (s *workerRuntimeSession) markFatal(msg string) {
s.mu.Lock()
defer s.mu.Unlock()
s.fatal = true
if s.fatalMsg == "" {
s.fatalMsg = msg
}
}
func (s *workerRuntimeSession) kill() {
if s.cmd != nil && s.cmd.Process != nil {
_ = s.cmd.Process.Kill()
}
_ = s.stdin.Close()
_ = s.stdout.Close()
}
func (s *workerRuntimeSession) withStderr(err error) error {
if err == nil {
return nil
}
stderr := strings.TrimSpace(s.stderr.String())
if stderr == "" {
return err
}
return fmt.Errorf("%w: worker stderr: %s", err, stderr)
}
func acquireWorkerSlot(ctx context.Context) (*semaphore.Weighted, error) {
timeout := envDuration("WANDERER_PLUGIN_WORKER_SLOT_TIMEOUT", defaultWorkerSlotAcquireTimeout)
acquireCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
slot := workerSemaphore()
if err := slot.Acquire(acquireCtx, 1); err != nil {
return nil, fmt.Errorf("acquire plugin worker slot: %w", err)
}
return slot, nil
}
func workerSemaphore() *semaphore.Weighted {
limit := int64(envInt("WANDERER_PLUGIN_WORKER_MAX", maxInt(2, runtime.NumCPU())))
workerSlotsMu.Lock()
defer workerSlotsMu.Unlock()
if workerSlots == nil {
workerSlots = semaphore.NewWeighted(limit)
}
return workerSlots
}
func envInt(key string, fallback int) int {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
return fallback
}
value, err := strconv.Atoi(raw)
if err != nil || value <= 0 {
return fallback
}
return value
}
func envDuration(key string, fallback time.Duration) time.Duration {
raw := strings.TrimSpace(os.Getenv(key))
if raw == "" {
return fallback
}
if value, err := time.ParseDuration(raw); err == nil && value > 0 {
return value
}
seconds, err := strconv.Atoi(raw)
if err != nil || seconds <= 0 {
return fallback
}
return time.Duration(seconds) * time.Second
}
func maxInt(a, b int) int {
if a > b {
return a
}
return b
}
func childEnvWithout(keys ...string) []string {
blocked := map[string]bool{}
for _, key := range keys {
blocked[key] = true
}
env := os.Environ()
filtered := make([]string, 0, len(env))
for _, entry := range env {
key := entry
if idx := strings.IndexByte(entry, '='); idx >= 0 {
key = entry[:idx]
}
if blocked[key] {
continue
}
filtered = append(filtered, entry)
}
return filtered
}
func newWorkerSessionID(pluginID string) string {
var random [8]byte
if _, err := rand.Read(random[:]); err != nil {
return pluginID + "-" + strconv.FormatInt(time.Now().UnixNano(), 10)
}
return pluginID + "-" + hex.EncodeToString(random[:])
}
type boundedWorkerBuffer struct {
mu sync.Mutex
limit int
data []byte
}
func (b *boundedWorkerBuffer) Write(p []byte) (int, error) {
b.mu.Lock()
defer b.mu.Unlock()
if b.limit <= 0 || len(b.data) >= b.limit {
return len(p), nil
}
remaining := b.limit - len(b.data)
if len(p) > remaining {
b.data = append(b.data, p[:remaining]...)
return len(p), nil
}
b.data = append(b.data, p...)
return len(p), nil
}
func (b *boundedWorkerBuffer) String() string {
b.mu.Lock()
defer b.mu.Unlock()
return string(b.data)
}

285
db/pluginsystem/worker_process.go Normal file
View File

@@ -0,0 +1,285 @@
package pluginsystem
import (
"context"
"encoding/json"
"fmt"
"io"
"os"
extism "github.com/extism/go-sdk"
)
// RunPluginWorker runs the stdio worker process. It is called by the main
// binary's plugin-worker subcommand before PocketBase is initialized.
func RunPluginWorker(ctx context.Context, stdin io.Reader, stdout io.Writer, stderr io.Writer) int {
_ = os.Unsetenv("EXTISM_ENABLE_WASI_OUTPUT")
worker := &pluginWorkerProcess{
ctx: ctx,
stdin: stdin,
stdout: stdout,
stderr: stderr,
requestMaxBytes: envInt("WANDERER_PLUGIN_WORKER_REQUEST_BYTES", defaultWorkerRequestMaxBytes),
responseMaxBytes: envInt("WANDERER_PLUGIN_WORKER_RESPONSE_BYTES", defaultWorkerResponseMaxBytes),
}
if err := worker.run(); err != nil {
_, _ = fmt.Fprintf(stderr, "plugin worker: %v\n", err)
return 1
}
return 0
}
type pluginWorkerProcess struct {
ctx context.Context
stdin io.Reader
stdout io.Writer
stderr io.Writer
requestMaxBytes int
responseMaxBytes int
wasmPath string
sessionID string
instance *extism.Plugin
fatalErr error
}
func (w *pluginWorkerProcess) run() error {
defer func() {
if w.instance != nil {
_ = w.instance.Close(w.ctx)
}
}()
for {
msg, err := readWorkerMessage(w.stdin, w.requestMaxBytes)
if err != nil {
// A clean io.EOF means the parent closed stdin without a
// shutdown frame (e.g. it crashed); exit quietly. An
// io.ErrUnexpectedEOF means stdin was cut mid-frame, which is a
// truncated/corrupt frame and should surface as an error.
if err == io.EOF {
return nil
}
return err
}
switch msg.Type {
case workerMessageShutdown:
return nil
case workerMessageCallExport:
if err := w.handleCallExport(msg); err != nil {
_ = w.sendError(err.Error())
return err
}
default:
err := fmt.Errorf("unexpected worker message %q", msg.Type)
_ = w.sendError(err.Error())
return err
}
}
}
func (w *pluginWorkerProcess) handleCallExport(msg workerMessage) error {
call, err := workerData[workerCallExport](msg)
if err != nil {
return err
}
if call.WASMPath == "" || call.Export == "" {
return fmt.Errorf("call_export requires wasmPath and export")
}
// The session ID is set by the parent once per worker process and reused
// for every call. It carries no routing semantics here (a worker serves a
// single wasm path) but is threaded into errors so captured stderr can be
// tied back to a specific session during diagnosis.
w.sessionID = call.SessionID
if w.instance == nil {
if err := w.openPlugin(call.WASMPath); err != nil {
return w.errCtx(call.Export, err)
}
} else if call.WASMPath != w.wasmPath {
return w.errCtx(call.Export, fmt.Errorf("worker session cannot switch wasm path (have %q, got %q)", w.wasmPath, call.WASMPath))
}
input, err := decodeWorkerBytes(call.InputBase64)
if err != nil {
return w.errCtx(call.Export, fmt.Errorf("decode call input: %w", err))
}
w.fatalErr = nil
code, output, err := w.instance.CallWithContext(w.ctx, call.Export, input)
if w.fatalErr != nil {
return w.errCtx(call.Export, w.fatalErr)
}
if err != nil {
return w.errCtx(call.Export, fmt.Errorf("call %s: %w", call.Export, err))
}
if code != 0 {
pluginErr := pluginErrorForCode(call.Export, code, w.instance.GetErrorWithContext(w.ctx))
return w.sendCallResult(workerCallResult{PluginError: &pluginErr})
}
return w.sendCallResult(workerCallResult{OutputBase64: encodeWorkerBytes(output)})
}
// pluginErrorForCode maps a non-zero export return code into the PluginError
// reported to the parent. It prefers the structured error JSON the plugin set
// via the host error API, and falls back to a generic plugin_error when that
// payload is missing, malformed, or has no code.
func pluginErrorForCode(export string, code uint32, rawErr string) PluginError {
var parsed PluginError
if rawErr == "" || json.Unmarshal([]byte(rawErr), &parsed) != nil || parsed.Code == "" {
return PluginError{
Code: "plugin_error",
Message: fmt.Sprintf("call %s failed with code %d", export, code),
}
}
return parsed
}
func (w *pluginWorkerProcess) openPlugin(wasmPath string) error {
manifest := extism.Manifest{
Wasm: []extism.Wasm{
extism.WasmFile{Path: wasmPath},
},
}
instance, err := extism.NewPlugin(w.ctx, manifest, extism.PluginConfig{
EnableWasi: true,
}, w.hostFunctions())
if err != nil {
return fmt.Errorf("create wasm plugin: %w", err)
}
w.wasmPath = wasmPath
w.instance = instance
return nil
}
func (w *pluginWorkerProcess) hostFunctions() []extism.HostFunction {
httpFn := extism.NewHostFunctionWithStack(
"http_request",
func(ctx context.Context, plugin *extism.CurrentPlugin, stack []uint64) {
requestBytes, err := plugin.ReadBytes(stack[0])
if err != nil {
writeHostHTTPResponse(ctx, plugin, stack, hostHTTPResponse{
Error: &PluginError{Code: "invalid_request", Message: err.Error()},
})
return
}
msg, err := workerMessageWithData(workerMessageHostHTTPRequest, workerHostHTTPRequest{
RequestBase64: encodeWorkerBytes(requestBytes),
})
if err != nil {
writeHostHTTPResponse(ctx, plugin, stack, hostHTTPResponse{
Error: &PluginError{Code: "internal_error", Message: err.Error()},
})
return
}
if err := writeWorkerMessage(w.stdout, w.responseMaxBytes, msg); err != nil {
w.failHostRPC(stack, fmt.Errorf("write host http request: %w", err))
return
}
responseMsg, err := readWorkerMessage(w.stdin, w.responseMaxBytes)
if err != nil {
w.failHostRPC(stack, fmt.Errorf("read host http response: %w", err))
return
}
if responseMsg.Type != workerMessageHostHTTPResponse {
w.failHostRPC(stack, fmt.Errorf("unexpected host http response message %q", responseMsg.Type))
return
}
response, err := workerData[workerHostHTTPResponse](responseMsg)
if err != nil {
w.failHostRPC(stack, fmt.Errorf("decode host http response: %w", err))
return
}
responseBytes, err := decodeWorkerBytes(response.ResponseBase64)
if err != nil {
w.failHostRPC(stack, fmt.Errorf("decode host http response bytes: %w", err))
return
}
offset, err := plugin.WriteBytes(responseBytes)
if err != nil {
plugin.Log(extism.LogLevelError, "write host http response: "+err.Error())
stack[0] = 0
return
}
stack[0] = offset
},
[]extism.ValueType{extism.ValueTypePTR},
[]extism.ValueType{extism.ValueTypePTR},
)
httpFn.SetNamespace("wanderer")
logFn := extism.NewHostFunctionWithStack(
"log",
func(ctx context.Context, plugin *extism.CurrentPlugin, stack []uint64) {
message, err := readBoundedHostLogPayload(plugin, stack[0])
if err != nil {
plugin.Log(extism.LogLevelError, "read host log message: "+err.Error())
return
}
entry, err := parseHostLogEntry(message)
if err != nil {
_, _ = fmt.Fprintf(w.stderr, "plugin log invalid: session %s: %v\n", w.sessionID, err)
return
}
msg, err := workerMessageWithData(workerMessageHostLog, workerHostLog{
Level: entry.Level,
Message: entry.Message,
SessionID: w.sessionID,
})
if err != nil {
_, _ = fmt.Fprintf(w.stderr, "plugin log encode failed: session %s: %v\n", w.sessionID, err)
return
}
if err := writeWorkerMessage(w.stdout, w.responseMaxBytes, msg); err != nil {
_, _ = fmt.Fprintf(w.stderr, "plugin log write failed: session %s: %v\n", w.sessionID, err)
}
_ = ctx
},
[]extism.ValueType{extism.ValueTypePTR},
nil,
)
logFn.SetNamespace("wanderer")
return []extism.HostFunction{httpFn, logFn}
}
func (w *pluginWorkerProcess) failHostRPC(stack []uint64, err error) {
w.fatalErr = err
stack[0] = 0
}
// errCtx annotates a fatal worker error with the active session and export so
// the message that the parent captures from stderr can be tied back to a
// specific call during diagnosis.
func (w *pluginWorkerProcess) errCtx(export string, err error) error {
if err == nil {
return nil
}
return fmt.Errorf("session %s export %s: %w", w.sessionID, export, err)
}
// Worker error channels follow a strict convention:
//
// - sendCallResult with a PluginError reports a business-level rejection from
// the plugin (a bad call code). The session stays alive and reusable; the
// parent surfaces it as a PluginCallError.
// - sendError reports a broken protocol or runtime (corrupt frame, host RPC
// failure, unexpected message). The parent treats it as fatal and tears the
// session down.
//
// Keep new failure paths on the correct channel: recoverable plugin outcomes
// use sendCallResult, anything that invalidates the session uses sendError.
func (w *pluginWorkerProcess) sendCallResult(result workerCallResult) error {
msg, err := workerMessageWithData(workerMessageCallResult, result)
if err != nil {
return err
}
return writeWorkerMessage(w.stdout, w.responseMaxBytes, msg)
}
func (w *pluginWorkerProcess) sendError(message string) error {
msg, err := workerMessageWithData(workerMessageError, workerError{Message: message})
if err != nil {
return err
}
return writeWorkerMessage(w.stdout, w.responseMaxBytes, msg)
}

149
db/pluginsystem/worker_rpc.go Normal file
View File

@@ -0,0 +1,149 @@
package pluginsystem
import (
"encoding/base64"
"encoding/binary"
"encoding/json"
"fmt"
"io"
)
const (
workerMessageCallExport = "call_export"
workerMessageShutdown = "shutdown"
workerMessageHostHTTPResponse = "host_http_response"
workerMessageHostHTTPRequest = "host_http_request"
workerMessageHostLog = "host_log"
workerMessageCallResult = "call_result"
workerMessageError = "error"
defaultWorkerRequestMaxBytes = 32 * 1024 * 1024
defaultWorkerResponseMaxBytes = 64 * 1024 * 1024
)
// workerMessage is one framed RPC message on the worker stdio protocol. The
// protocol is strictly synchronous (one call_export in flight at a time, with
// host HTTP RPC nested synchronously), so messages carry no correlation ID.
type workerMessage struct {
Type string `json:"type"`
Data json.RawMessage `json:"data,omitempty"`
}
type workerCallExport struct {
WASMPath string `json:"wasmPath"`
Export string `json:"export"`
InputBase64 string `json:"inputBase64,omitempty"`
SessionID string `json:"sessionId,omitempty"`
}
type workerCallResult struct {
OutputBase64 string `json:"outputBase64,omitempty"`
PluginError *PluginError `json:"pluginError,omitempty"`
}
type workerHostHTTPRequest struct {
RequestBase64 string `json:"requestBase64"`
}
type workerHostHTTPResponse struct {
ResponseBase64 string `json:"responseBase64"`
}
type workerHostLog struct {
Level string `json:"level"`
Message string `json:"message"`
SessionID string `json:"sessionId,omitempty"`
}
type workerError struct {
Message string `json:"message"`
}
func writeWorkerMessage(w io.Writer, maxBytes int, msg workerMessage) error {
payload, err := json.Marshal(msg)
if err != nil {
return err
}
if len(payload) > maxBytes {
return fmt.Errorf("worker rpc frame too large: %d > %d", len(payload), maxBytes)
}
var header [4]byte
binary.BigEndian.PutUint32(header[:], uint32(len(payload)))
if err := writeAll(w, header[:]); err != nil {
return err
}
return writeAll(w, payload)
}
func readWorkerMessage(r io.Reader, maxBytes int) (workerMessage, error) {
var header [4]byte
if _, err := io.ReadFull(r, header[:]); err != nil {
return workerMessage{}, err
}
size := binary.BigEndian.Uint32(header[:])
if size == 0 {
return workerMessage{}, fmt.Errorf("worker rpc frame is empty")
}
if int(size) > maxBytes {
return workerMessage{}, fmt.Errorf("worker rpc frame too large: %d > %d", size, maxBytes)
}
payload := make([]byte, int(size))
if _, err := io.ReadFull(r, payload); err != nil {
return workerMessage{}, err
}
var msg workerMessage
if err := json.Unmarshal(payload, &msg); err != nil {
return workerMessage{}, err
}
if msg.Type == "" {
return workerMessage{}, fmt.Errorf("worker rpc message type is empty")
}
return msg, nil
}
func encodeWorkerBytes(data []byte) string {
if len(data) == 0 {
return ""
}
return base64.StdEncoding.EncodeToString(data)
}
func decodeWorkerBytes(encoded string) ([]byte, error) {
if encoded == "" {
return nil, nil
}
return base64.StdEncoding.DecodeString(encoded)
}
func workerData[T any](msg workerMessage) (T, error) {
var value T
if len(msg.Data) == 0 {
return value, nil
}
if err := json.Unmarshal(msg.Data, &value); err != nil {
return value, err
}
return value, nil
}
func workerMessageWithData[T any](typ string, data T) (workerMessage, error) {
raw, err := json.Marshal(data)
if err != nil {
return workerMessage{}, err
}
return workerMessage{Type: typ, Data: raw}, nil
}
func writeAll(w io.Writer, data []byte) error {
for len(data) > 0 {
n, err := w.Write(data)
if err != nil {
return err
}
if n == 0 {
return io.ErrShortWrite
}
data = data[n:]
}
return nil
}

369
db/pluginsystem/worker_test.go Normal file
View File

@@ -0,0 +1,369 @@
package pluginsystem
import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"os"
"strings"
"testing"
extism "github.com/extism/go-sdk"
"github.com/pocketbase/pocketbase/core"
)
func TestWorkerRPCFrameRoundTrip(t *testing.T) {
msg, err := workerMessageWithData(workerMessageCallExport, workerCallExport{
WASMPath: "/tmp/plugin.wasm",
Export: "list_routes_v1",
InputBase64: encodeWorkerBytes([]byte(`{"ok":true}`)),
})
if err != nil {
t.Fatal(err)
}
var buf bytes.Buffer
if err := writeWorkerMessage(&buf, 1024, msg); err != nil {
t.Fatalf("write message: %v", err)
}
got, err := readWorkerMessage(&buf, 1024)
if err != nil {
t.Fatalf("read message: %v", err)
}
if got.Type != workerMessageCallExport {
t.Fatalf("unexpected type: %q", got.Type)
}
payload, err := workerData[workerCallExport](got)
if err != nil {
t.Fatalf("decode payload: %v", err)
}
input, err := decodeWorkerBytes(payload.InputBase64)
if err != nil {
t.Fatalf("decode input: %v", err)
}
if string(input) != `{"ok":true}` {
t.Fatalf("unexpected input: %s", input)
}
}
func TestWorkerRPCRejectsOversizedFrameBeforePayloadRead(t *testing.T) {
msg, err := workerMessageWithData(workerMessageError, workerError{Message: "too large"})
if err != nil {
t.Fatal(err)
}
var buf bytes.Buffer
if err := writeWorkerMessage(&buf, 1024, msg); err != nil {
t.Fatalf("write message: %v", err)
}
if _, err := readWorkerMessage(&buf, 4); err == nil {
t.Fatal("expected oversized frame error")
}
}
func TestPluginWorkerExitsOnStdinEOF(t *testing.T) {
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunPluginWorker(context.Background(), bytes.NewReader(nil), &stdout, &stderr)
if code != 0 {
t.Fatalf("unexpected exit code %d, stderr %q", code, stderr.String())
}
if stdout.Len() != 0 {
t.Fatalf("unexpected stdout: %q", stdout.String())
}
}
func TestPluginWorkerTruncatedFrameReturnsError(t *testing.T) {
var header [4]byte
binary.BigEndian.PutUint32(header[:], 100)
stdin := bytes.NewReader(append(header[:], []byte("partial")...))
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunPluginWorker(context.Background(), stdin, &stdout, &stderr)
if code == 0 {
t.Fatal("expected non-zero exit code for truncated frame")
}
if stderr.Len() == 0 {
t.Fatal("expected truncated frame error on stderr")
}
}
func TestPluginWorkerUnexpectedMessageTypeFails(t *testing.T) {
msg, err := workerMessageWithData(workerMessageHostHTTPResponse, workerHostHTTPResponse{})
if err != nil {
t.Fatal(err)
}
var stdin bytes.Buffer
if err := writeWorkerMessage(&stdin, defaultWorkerRequestMaxBytes, msg); err != nil {
t.Fatal(err)
}
var stdout bytes.Buffer
var stderr bytes.Buffer
code := RunPluginWorker(context.Background(), &stdin, &stdout, &stderr)
if code == 0 {
t.Fatal("expected non-zero exit code for unexpected message type")
}
reply, err := readWorkerMessage(&stdout, defaultWorkerResponseMaxBytes)
if err != nil {
t.Fatalf("read worker reply: %v", err)
}
if reply.Type != workerMessageError {
t.Fatalf("expected error reply, got %q", reply.Type)
}
}
func TestHandleCallExportRejectsWasmPathSwitch(t *testing.T) {
worker := &pluginWorkerProcess{
ctx: context.Background(),
instance: &extism.Plugin{},
wasmPath: "/plugins/a.wasm",
}
msg, err := workerMessageWithData(workerMessageCallExport, workerCallExport{
WASMPath: "/plugins/b.wasm",
Export: "list_routes_v1",
SessionID: "sess-1",
})
if err != nil {
t.Fatal(err)
}
err = worker.handleCallExport(msg)
if err == nil {
t.Fatal("expected error when switching wasm path")
}
for _, want := range []string{"sess-1", "list_routes_v1", "/plugins/a.wasm", "/plugins/b.wasm"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error %q missing %q", err.Error(), want)
}
}
}
func TestHandleCallExportRejectsInvalidInput(t *testing.T) {
worker := &pluginWorkerProcess{
ctx: context.Background(),
instance: &extism.Plugin{},
wasmPath: "/plugins/a.wasm",
}
msg, err := workerMessageWithData(workerMessageCallExport, workerCallExport{
WASMPath: "/plugins/a.wasm",
Export: "list_routes_v1",
InputBase64: "!!!not-base64!!!",
SessionID: "sess-2",
})
if err != nil {
t.Fatal(err)
}
err = worker.handleCallExport(msg)
if err == nil {
t.Fatal("expected error for invalid input base64")
}
if !strings.Contains(err.Error(), "sess-2") || !strings.Contains(err.Error(), "decode call input") {
t.Fatalf("unexpected error: %v", err)
}
}
func TestWorkerHostLogFrameRoundTrip(t *testing.T) {
msg, err := workerMessageWithData(workerMessageHostLog, workerHostLog{
Level: "info",
Message: "detail fetch took 1s",
SessionID: "sess-log",
})
if err != nil {
t.Fatal(err)
}
var buf bytes.Buffer
if err := writeWorkerMessage(&buf, 1024, msg); err != nil {
t.Fatalf("write message: %v", err)
}
got, err := readWorkerMessage(&buf, 1024)
if err != nil {
t.Fatalf("read worker message: %v", err)
}
if got.Type != workerMessageHostLog {
t.Fatalf("expected host_log, got %q", got.Type)
}
payload, err := workerData[workerHostLog](got)
if err != nil {
t.Fatalf("decode host log: %v", err)
}
if payload.Level != "info" || payload.Message != "detail fetch took 1s" || payload.SessionID != "sess-log" {
t.Fatalf("unexpected host log payload: %#v", payload)
}
}
func TestPluginErrorForCode(t *testing.T) {
t.Run("falls back when raw error is empty", func(t *testing.T) {
got := pluginErrorForCode("list_routes_v1", 7, "")
if got.Code != "plugin_error" || !strings.Contains(got.Message, "code 7") {
t.Fatalf("unexpected fallback error: %#v", got)
}
})
t.Run("falls back when raw error is malformed", func(t *testing.T) {
got := pluginErrorForCode("list_routes_v1", 1, "{not json")
if got.Code != "plugin_error" {
t.Fatalf("expected fallback for malformed json, got %#v", got)
}
})
t.Run("falls back when code is empty", func(t *testing.T) {
got := pluginErrorForCode("list_routes_v1", 1, `{"message":"boom"}`)
if got.Code != "plugin_error" {
t.Fatalf("expected fallback for missing code, got %#v", got)
}
})
t.Run("passes through structured error", func(t *testing.T) {
got := pluginErrorForCode("list_routes_v1", 1, `{"code":"rate_limited","message":"slow down"}`)
if got.Code != "rate_limited" || got.Message != "slow down" {
t.Fatalf("expected structured error, got %#v", got)
}
})
}
func TestExecuteHostHTTPRequestRejectsInvalidPayload(t *testing.T) {
response := executeHostHTTPRequest(context.Background(), Manifest{}, RequestPolicyContext{}, []byte("not json"))
if response.Error == nil || response.Error.Code != "invalid_request" {
t.Fatalf("expected invalid_request error, got %#v", response)
}
}
func TestPluginWorkerHostRPCFatalSetsClearError(t *testing.T) {
worker := &pluginWorkerProcess{}
stack := []uint64{123}
worker.failHostRPC(stack, errors.New("host RPC read failed"))
if stack[0] != 0 {
t.Fatalf("expected null response pointer, got %d", stack[0])
}
if worker.fatalErr == nil || worker.fatalErr.Error() != "host RPC read failed" {
t.Fatalf("unexpected fatal error: %v", worker.fatalErr)
}
}
func TestRuntimeSessionFatalErrorIsDetectableThroughWrapping(t *testing.T) {
err := fmt.Errorf("outer: %w", RuntimeSessionFatalError{Err: errors.New("worker died")})
if !IsRuntimeSessionFatalError(err) {
t.Fatal("expected fatal session error")
}
if IsRuntimeSessionFatalError(errors.New("plugin error")) {
t.Fatal("unexpected fatal session error")
}
}
func TestChildEnvWithoutStripsKeys(t *testing.T) {
t.Setenv("EXTISM_ENABLE_WASI_OUTPUT", "1")
t.Setenv("WANDERER_TEST_KEEP", "yes")
env := childEnvWithout("EXTISM_ENABLE_WASI_OUTPUT")
for _, entry := range env {
if entry == "EXTISM_ENABLE_WASI_OUTPUT=1" {
t.Fatalf("unexpected stripped env entry in %#v", env)
}
}
if os.Getenv("EXTISM_ENABLE_WASI_OUTPUT") != "1" {
t.Fatal("childEnvWithout should not mutate the current process env")
}
}
func TestInjectHostRequestAuthUsesExistingSessionForRefresh(t *testing.T) {
spec := HostRequestSpec{Auth: "session"}
session := &fakeRuntimeSession{
output: []byte(`{"token":"session-token"}`),
}
err := InjectHostRequestAuth(context.Background(), AuthInjectionInput{
Session: session,
Plugin: LocalPlugin{Manifest: Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"session": {
Type: AuthTypeSession,
SecretFields: []string{"email", "password"},
Refresh: &AuthRefresh{Mode: AuthRefreshModePlugin, Function: "refresh_session_v1"},
},
}},
Permissions: PermissionManifest{Auth: []string{"session"}},
}},
Instance: testPluginInstance("inst1", "plugin.test"),
Auth: map[string]any{"email": "user@example.com", "password": "secret"},
Spec: &spec,
})
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if session.export != "refresh_session_v1" {
t.Fatalf("unexpected export: %q", session.export)
}
if got := spec.Headers[AuthHeaderAuthorization]; got != AuthSchemeBearer+" session-token" {
t.Fatalf("unexpected auth header: %q", got)
}
var input map[string]any
if err := json.Unmarshal(session.input, &input); err != nil {
t.Fatalf("invalid refresh input: %v", err)
}
auth, ok := input["auth"].(map[string]any)
if !ok {
t.Fatalf("missing refresh auth: %#v", input)
}
if _, ok := auth["accessToken"]; ok {
t.Fatalf("refresh auth leaked access token: %#v", auth)
}
}
type fakeRuntimeSession struct {
export string
input []byte
output []byte
err error
}
func (s *fakeRuntimeSession) Call(_ context.Context, export string, input []byte) ([]byte, error) {
s.export = export
s.input = append([]byte(nil), input...)
if s.err != nil {
return nil, s.err
}
return s.output, nil
}
func (s *fakeRuntimeSession) Close(context.Context) error {
return nil
}
func TestInjectHostRequestAuthDoesNotRequireRuntimeWhenSessionProvided(t *testing.T) {
spec := HostRequestSpec{Auth: "session"}
session := &fakeRuntimeSession{err: errors.New("session failed")}
err := InjectHostRequestAuth(context.Background(), AuthInjectionInput{
Session: session,
Plugin: LocalPlugin{Manifest: Manifest{
Auth: AuthManifest{Contexts: map[string]AuthContext{
"session": {
Type: AuthTypeSession,
SecretFields: []string{"email"},
Refresh: &AuthRefresh{Mode: AuthRefreshModePlugin, Function: "refresh_session_v1"},
},
}},
Permissions: PermissionManifest{Auth: []string{"session"}},
}},
Instance: testPluginInstance("inst1", "plugin.test"),
Auth: map[string]any{"email": "user@example.com"},
Spec: &spec,
})
if err == nil || err.Error() != "session failed" {
t.Fatalf("unexpected error: %v", err)
}
}
func testPluginInstance(id string, pluginID string) *core.Record {
collection := core.NewBaseCollection("plugin_instances")
collection.Fields.Add(&core.TextField{Name: "plugin_id"})
record := core.NewRecord(collection)
record.Id = id
record.Set("plugin_id", pluginID)
return record
}

81
db/routes/integration_hammerhead.go
View File

@@ -1,81 +0,0 @@
package routes
import (
"encoding/json"
"net/http"
"os"
"pocketbase/integrations/hammerhead"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
)
func IntegrationHammerheadUpload(e *core.RequestEvent) error {
h, err := loginHammerhead(e)
if err != nil {
return err
}
if err := h.UploadActivities(e); err != nil {
return err
}
return e.JSON(http.StatusOK, nil)
}
func IntegrationHammerheadLogin(e *core.RequestEvent) error {
_, err := loginHammerhead(e)
if err != nil {
return err
}
return e.JSON(http.StatusOK, nil)
}
func loginHammerhead(e *core.RequestEvent) (*hammerhead.HammerheadApi, error) {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return nil, apis.NewBadRequestError("POCKETBASE_ENCRYPTION_KEY not set", nil)
}
userId := ""
if e.Auth != nil {
userId = e.Auth.Id
} else {
return nil, e.UnauthorizedError("authentication required", nil)
}
integrations, err := e.App.FindAllRecords("integrations", dbx.NewExp("user = {:id}", dbx.Params{"id": userId}))
if err != nil {
return nil, err
}
if len(integrations) == 0 {
return nil, apis.NewBadRequestError("user has no integration", nil)
}
integration := integrations[0]
hammerheadString := integration.GetString("hammerhead")
if len(hammerheadString) == 0 {
return nil, apis.NewBadRequestError("hammerhead integration missing", nil)
}
var hammerheadIntegration hammerhead.HammerheadIntegration
err = json.Unmarshal([]byte(hammerheadString), &hammerheadIntegration)
if err != nil {
return nil, err
}
decryptedPassword, err := security.Decrypt(hammerheadIntegration.Password, encryptionKey)
if err != nil {
return nil, err
}
k := &hammerhead.HammerheadApi{}
err = k.Login(hammerheadIntegration.Email, string(decryptedPassword))
if err != nil {
return nil, apis.NewUnauthorizedError("invalid credentials", nil)
}
return k, e.JSON(http.StatusOK, nil)
}

58
db/routes/integration_komoot.go
View File

@@ -1,58 +0,0 @@
package routes
import (
"encoding/json"
"net/http"
"os"
"pocketbase/integrations/komoot"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
)
func IntegrationKommotLogin(e *core.RequestEvent) error {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return apis.NewBadRequestError("POCKETBASE_ENCRYPTION_KEY not set", nil)
}
userId := ""
if e.Auth != nil {
userId = e.Auth.Id
} else {
return e.UnauthorizedError("authentication required", nil)
}
integrations, err := e.App.FindAllRecords("integrations", dbx.NewExp("user = {:id}", dbx.Params{"id": userId}))
if err != nil {
return err
}
if len(integrations) == 0 {
return apis.NewBadRequestError("user has no integration", nil)
}
integration := integrations[0]
komootString := integration.GetString("komoot")
if len(komootString) == 0 {
return apis.NewBadRequestError("komoot integration missing", nil)
}
var komootIntegration komoot.KomootIntegration
err = json.Unmarshal([]byte(komootString), &komootIntegration)
if err != nil {
return err
}
decryptedPassword, err := security.Decrypt(komootIntegration.Password, encryptionKey)
if err != nil {
return err
}
k := &komoot.KomootApi{}
err = k.Login(komootIntegration.Email, string(decryptedPassword))
if err != nil {
return apis.NewUnauthorizedError("invalid credentials", nil)
}
return e.JSON(http.StatusOK, nil)
}

87
db/routes/integration_strava.go
View File

@@ -1,87 +0,0 @@
package routes
import (
"encoding/json"
"net/http"
"os"
"pocketbase/integrations/strava"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
)
func IntegrationStravaToken(e *core.RequestEvent) error {
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if len(encryptionKey) == 0 {
return apis.NewBadRequestError("POCKETBASE_ENCRYPTION_KEY not set", nil)
}
var data strava.TokenRequest
if err := e.BindBody(&data); err != nil {
return apis.NewBadRequestError("Failed to read request data", err)
}
userId := ""
if e.Auth != nil {
userId = e.Auth.Id
} else {
return e.UnauthorizedError("authentication required", nil)
}
integrations, err := e.App.FindAllRecords("integrations", dbx.NewExp("user = {:id}", dbx.Params{"id": userId}))
if err != nil {
return err
}
if len(integrations) == 0 {
return apis.NewBadRequestError("user has no integration", nil)
}
integration := integrations[0]
stravaString := integration.GetString("strava")
if len(stravaString) == 0 {
return apis.NewBadRequestError("strava integration missing", nil)
}
var stravaIntegration strava.StravaIntegration
err = json.Unmarshal([]byte(stravaString), &stravaIntegration)
if err != nil {
return err
}
decryptedSecret, err := security.Decrypt(stravaIntegration.ClientSecret, encryptionKey)
if err != nil {
return err
}
request := strava.TokenRequest{
ClientID: stravaIntegration.ClientID,
ClientSecret: string(decryptedSecret),
Code: data.Code,
GrantType: "authorization_code",
}
r, err := strava.GetStravaToken(request)
if err != nil {
return err
}
if r.AccessToken != "" {
stravaIntegration.AccessToken = r.AccessToken
}
if r.RefreshToken != "" {
stravaIntegration.RefreshToken = r.RefreshToken
}
if r.AccessToken != "" {
stravaIntegration.ExpiresAt = r.ExpiresAt
}
stravaIntegration.Active = true
b, err := json.Marshal(stravaIntegration)
if err != nil {
return err
}
integration.Set("strava", string(b))
err = e.App.Save(integration)
if err != nil {
return err
}
return e.JSON(http.StatusOK, nil)
}

72
db/routes/plugin_system.go Normal file
View File

@@ -0,0 +1,72 @@
package routes
import (
"net/http"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"pocketbase/pluginsystem"
)
// PluginSystemPluginsList refreshes the installed plugin cache and returns the
// plugins that are available from the local runtime directory.
func PluginSystemPluginsList(e *core.RequestEvent) error {
if e.Auth == nil && !e.HasSuperuserAuth() {
return apis.NewUnauthorizedError("authentication required", nil)
}
manager := pluginsystem.NewManager(e.App, "")
if err := manager.SyncInstalledPlugins(e.Request.Context()); err != nil {
return err
}
plugins, err := manager.ListLocalPlugins(e.Request.Context())
if err != nil {
return err
}
if !e.HasSuperuserAuth() {
for i := range plugins {
plugins[i].Path = ""
}
}
return e.JSON(http.StatusOK, map[string]any{"items": plugins})
}
// localPlugin resolves an installed plugin from the cached installed_plugins
// record, with disk manifest fallback handled inside pluginsystem.
func localPlugin(app core.App, pluginID string) (pluginsystem.LocalPlugin, error) {
plugin, err := pluginsystem.LoadInstalledPlugin(app, "", pluginID)
if err != nil {
return pluginsystem.LocalPlugin{}, apis.NewBadRequestError("unknown plugin", err)
}
return plugin, nil
}
// pluginCapability returns the manifest entry for a concrete capability/version
// pair so the host can call the export declared by the plugin.
func pluginCapability(plugin pluginsystem.LocalPlugin, name string, version string) (pluginsystem.CapabilityManifest, error) {
for _, capability := range plugin.Manifest.Capabilities {
if capability.Name == name && capability.Version == version {
return capability, nil
}
}
return pluginsystem.CapabilityManifest{}, apis.NewBadRequestError("plugin capability is not available", map[string]string{
"name": name,
"version": version,
})
}
// localPluginCapability resolves an installed plugin and verifies that it
// declares the requested capability.
func localPluginCapability(app core.App, pluginID string, name string, version string) (pluginsystem.LocalPlugin, pluginsystem.CapabilityManifest, error) {
plugin, err := localPlugin(app, pluginID)
if err != nil {
return pluginsystem.LocalPlugin{}, pluginsystem.CapabilityManifest{}, err
}
capability, err := pluginCapability(plugin, name, version)
if err != nil {
return pluginsystem.LocalPlugin{}, pluginsystem.CapabilityManifest{}, err
}
return plugin, capability, nil
}

222
db/routes/plugin_system_auth.go Normal file
View File

@@ -0,0 +1,222 @@
package routes
import (
"net/http"
"net/url"
"strings"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"pocketbase/pluginsystem"
)
type pluginOAuthStartRequest struct {
PluginID string `json:"pluginId"`
InstanceID string `json:"instanceId"`
AuthContext string `json:"authContext,omitempty"`
RedirectURI string `json:"redirectUri"`
}
type pluginOAuthCallbackRequest struct {
InstanceID string `json:"instanceId"`
Code string `json:"code"`
State string `json:"state"`
}
type pluginOAuthRevokeRequest struct {
InstanceID string `json:"instanceId"`
}
func PluginSystemOAuthStart(e *core.RequestEvent) error {
if e.Auth == nil {
return apis.NewUnauthorizedError("authentication required", nil)
}
var data pluginOAuthStartRequest
if err := e.BindBody(&data); err != nil {
return apis.NewBadRequestError("failed to read request data", err)
}
if data.PluginID == "" || data.RedirectURI == "" {
return apis.NewBadRequestError("pluginId and redirectUri are required", nil)
}
if err := pluginsystem.ValidateOAuthRedirectURI(data.RedirectURI); err != nil {
return apis.NewBadRequestError("redirectUri is not allowed", err)
}
plugin, err := localPlugin(e.App, data.PluginID)
if err != nil {
return err
}
contextName, authContext, err := pluginsystem.OAuthContext(plugin, data.AuthContext)
if err != nil {
return apis.NewBadRequestError("plugin has no oauth auth context", err)
}
instance, err := pluginAuthInstance(e.App, e.Auth.Id, data.PluginID, data.InstanceID)
if err != nil {
return err
}
auth, err := decryptedInstanceAuth(instance)
if err != nil {
return err
}
clientID := pluginsystem.StringFromAny(auth["clientId"])
if clientID == "" {
return apis.NewBadRequestError("oauth clientId is required", nil)
}
state := pluginsystem.NewOAuthState(32)
auth[pluginsystem.AuthFieldOAuthContext] = contextName
auth[pluginsystem.AuthFieldOAuthState] = state
auth[pluginsystem.AuthFieldOAuthRedirectURI] = data.RedirectURI
values := url.Values{}
values.Set("response_type", "code")
values.Set("client_id", clientID)
values.Set("redirect_uri", data.RedirectURI)
values.Set("state", state)
if len(authContext.Scopes) > 0 {
separator := authContext.ScopeSeparator
if separator == "" {
separator = " "
}
values.Set("scope", strings.Join(authContext.Scopes, separator))
}
for key, value := range authContext.AuthorizationParams {
values.Set(key, value)
}
if authContext.PKCE {
verifier := pluginsystem.NewOAuthCodeVerifier(64)
auth[pluginsystem.AuthFieldOAuthCodeVerifier] = verifier
values.Set("code_challenge_method", "S256")
values.Set("code_challenge", pluginsystem.PKCEChallenge(verifier))
}
instance.Set("auth", auth)
instance.Set("status", "needs_auth")
if err := e.App.Save(instance); err != nil {
return err
}
authURL, err := url.Parse(authContext.AuthorizationURL)
if err != nil {
return err
}
query := authURL.Query()
for key, value := range values {
query[key] = value
}
authURL.RawQuery = query.Encode()
return e.JSON(http.StatusOK, map[string]any{
"url": authURL.String(),
"state": state,
"instanceId": instance.Id,
})
}
func PluginSystemOAuthCallback(e *core.RequestEvent) error {
if e.Auth == nil {
return apis.NewUnauthorizedError("authentication required", nil)
}
var data pluginOAuthCallbackRequest
if err := e.BindBody(&data); err != nil {
return apis.NewBadRequestError("failed to read request data", err)
}
if data.InstanceID == "" || data.Code == "" || data.State == "" {
return apis.NewBadRequestError("instanceId, code and state are required", nil)
}
instance, err := e.App.FindRecordById("plugin_instances", data.InstanceID)
if err != nil || instance.GetString("user") != e.Auth.Id {
return apis.NewNotFoundError("plugin instance not found", nil)
}
plugin, err := localPlugin(e.App, instance.GetString("plugin_id"))
if err != nil {
return err
}
auth, err := decryptedInstanceAuth(instance)
if err != nil {
return err
}
if data.State != pluginsystem.StringFromAny(auth[pluginsystem.AuthFieldOAuthState]) {
return apis.NewBadRequestError("invalid oauth state", nil)
}
contextName := pluginsystem.StringFromAny(auth[pluginsystem.AuthFieldOAuthContext])
_, authContext, err := pluginsystem.OAuthContext(plugin, contextName)
if err != nil {
return apis.NewBadRequestError("plugin has no oauth auth context", err)
}
token, err := pluginsystem.ExchangeOAuthToken(e.Request.Context(), plugin.Manifest, authContext, auth, map[string]string{
"grant_type": "authorization_code",
"code": data.Code,
"redirect_uri": pluginsystem.StringFromAny(auth[pluginsystem.AuthFieldOAuthRedirectURI]),
"code_verifier": pluginsystem.StringFromAny(
auth[pluginsystem.AuthFieldOAuthCodeVerifier],
),
})
if err != nil {
return apis.NewBadRequestError("oauth token exchange failed", err)
}
pluginsystem.StoreOAuthToken(auth, contextName, token)
for _, field := range pluginsystem.InternalOAuthTransientFields() {
delete(auth, field)
}
instance.Set("auth", auth)
instance.Set("status", "configured")
instance.Set("last_error", map[string]any{})
if err := e.App.Save(instance); err != nil {
return err
}
return e.JSON(http.StatusOK, map[string]any{"ok": true})
}
func PluginSystemOAuthRevoke(e *core.RequestEvent) error {
if e.Auth == nil {
return apis.NewUnauthorizedError("authentication required", nil)
}
var data pluginOAuthRevokeRequest
if err := e.BindBody(&data); err != nil {
return apis.NewBadRequestError("failed to read request data", err)
}
if data.InstanceID == "" {
return apis.NewBadRequestError("instanceId is required", nil)
}
instance, err := e.App.FindRecordById("plugin_instances", data.InstanceID)
if err != nil || instance.GetString("user") != e.Auth.Id {
return apis.NewNotFoundError("plugin instance not found", nil)
}
auth, err := decryptedInstanceAuth(instance)
if err != nil {
return err
}
pluginsystem.ClearOAuthToken(auth)
instance.Set("auth", auth)
instance.Set("status", "needs_auth")
if err := e.App.Save(instance); err != nil {
return err
}
return e.JSON(http.StatusOK, map[string]any{"ok": true})
}
func pluginAuthInstance(app core.App, userID string, pluginID string, instanceID string) (*core.Record, error) {
if instanceID != "" {
instance, err := app.FindRecordById("plugin_instances", instanceID)
if err != nil || instance.GetString("user") != userID || instance.GetString("plugin_id") != pluginID {
return nil, apis.NewNotFoundError("plugin instance not found", nil)
}
return instance, nil
}
return app.FindFirstRecordByFilter(
"plugin_instances",
"user={:user} && plugin_id={:plugin_id}",
dbx.Params{"user": userID, "plugin_id": pluginID},
)
}

242
db/routes/plugin_system_category_remap.go Normal file
View File

@@ -0,0 +1,242 @@
package routes
import (
"net/http"
"strings"
"time"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"pocketbase/plugins/importer"
)
type pluginCategoryRemapRequest struct {
InstanceID string `json:"instanceId"`
Config map[string]any `json:"config,omitempty"`
}
type pluginCategoryRemapResponse struct {
Count int `json:"count"`
BackfilledSinceMapping int `json:"backfilledSinceMapping,omitempty"`
Remapped int `json:"remapped,omitempty"`
}
type pluginCategoryRemapCandidate struct {
Trail *core.Record
CategoryID string
}
type pluginCategoryTrailReference struct {
Ref *core.Record
Trail *core.Record
ExternalID string
}
// PluginSystemCategoryRemapPreview counts imported trails whose stored provider
// category can be mapped with the current plugin instance configuration.
func PluginSystemCategoryRemapPreview(e *core.RequestEvent) error {
instance, mapping, err := pluginCategoryRemapInput(e)
if err != nil {
return err
}
refs, err := pluginCategoryTrailReferences(e.App, e.Auth.Id, instance.GetString("plugin_id"))
if err != nil {
return err
}
candidates := pluginCategoryRemapCandidatesFromRefs(e.App, refs, mapping)
backfilledSinceMapping := pluginCategoryBackfilledSinceMappingCountFromRefs(e.App, instance, refs, mapping)
return e.JSON(http.StatusOK, pluginCategoryRemapResponse{
Count: len(candidates),
BackfilledSinceMapping: backfilledSinceMapping,
})
}
// PluginSystemCategoryRemapApply updates the local category of imported trails
// whose stored provider category matches the current plugin instance mapping.
func PluginSystemCategoryRemapApply(e *core.RequestEvent) error {
instance, mapping, err := pluginCategoryRemapInput(e)
if err != nil {
return err
}
candidates, err := pluginCategoryRemapCandidates(e.App, e.Auth.Id, instance.GetString("plugin_id"), mapping)
if err != nil {
return err
}
remapped := 0
if err := e.App.RunInTransaction(func(txApp core.App) error {
for _, candidate := range candidates {
trail, err := txApp.FindRecordById("trails", candidate.Trail.Id)
if err != nil {
return err
}
trail.Set("category", candidate.CategoryID)
if err := txApp.Save(trail); err != nil {
return err
}
remapped++
}
return nil
}); err != nil {
return err
}
return e.JSON(http.StatusOK, pluginCategoryRemapResponse{Count: len(candidates), Remapped: remapped})
}
func pluginCategoryRemapInput(e *core.RequestEvent) (*core.Record, map[string]string, error) {
if e.Auth == nil {
return nil, nil, apis.NewUnauthorizedError("authentication required", nil)
}
var data pluginCategoryRemapRequest
if err := e.BindBody(&data); err != nil {
return nil, nil, apis.NewBadRequestError("Failed to read request data", err)
}
if data.InstanceID == "" {
return nil, nil, apis.NewBadRequestError("instanceId is required", nil)
}
instance, err := e.App.FindRecordById("plugin_instances", data.InstanceID)
if err != nil || instance.GetString("user") != e.Auth.Id {
return nil, nil, apis.NewNotFoundError("plugin instance not found", err)
}
config := effectivePluginConfig(e.App, instance.GetString("plugin_id"), instance)
if data.Config != nil {
config = data.Config
}
return instance, categoryMapping(pluginHostConfig(config)), nil
}
func pluginCategoryRemapCandidates(app core.App, userID string, pluginID string, mapping map[string]string) ([]pluginCategoryRemapCandidate, error) {
if userID == "" || pluginID == "" || len(mapping) == 0 {
return nil, nil
}
refs, err := pluginCategoryTrailReferences(app, userID, pluginID)
if err != nil || len(refs) == 0 {
return nil, err
}
return pluginCategoryRemapCandidatesFromRefs(app, refs, mapping), nil
}
func pluginCategoryRemapCandidatesFromRefs(app core.App, refs []pluginCategoryTrailReference, mapping map[string]string) []pluginCategoryRemapCandidate {
if len(refs) == 0 || len(mapping) == 0 {
return nil
}
candidates := make([]pluginCategoryRemapCandidate, 0, len(refs))
for _, ref := range refs {
providerCategory := strings.TrimSpace(ref.Ref.GetString("provider_category"))
categoryID, matched := importer.CategoryFromProviderMapping(app, providerCategory, mapping)
if !matched || categoryID == "" || ref.Trail.GetString("category") == categoryID {
continue
}
candidates = append(candidates, pluginCategoryRemapCandidate{
Trail: ref.Trail,
CategoryID: categoryID,
})
}
return candidates
}
func pluginCategoryBackfilledSinceMappingCountFromRefs(app core.App, instance *core.Record, refs []pluginCategoryTrailReference, mapping map[string]string) int {
mappingUpdatedAt := categoryMappingUpdatedAt(app, instance)
if mappingUpdatedAt.IsZero() || len(refs) == 0 || len(mapping) == 0 {
return 0
}
count := 0
for _, ref := range refs {
checkedAt := ref.Ref.GetDateTime("provider_category_checked_at")
if checkedAt.IsZero() || !checkedAt.Time().After(mappingUpdatedAt) {
continue
}
providerCategory := strings.TrimSpace(ref.Ref.GetString("provider_category"))
categoryID, matched := importer.CategoryFromProviderMapping(app, providerCategory, mapping)
if matched && categoryID != "" && ref.Trail.GetString("category") != categoryID {
count++
}
}
return count
}
func categoryMappingUpdatedAt(app core.App, instance *core.Record) time.Time {
if instance == nil {
return time.Time{}
}
config := effectivePluginConfig(app, instance.GetString("plugin_id"), instance)
raw, _ := pluginHostConfig(config)["categoryMappingUpdatedAt"].(string)
if raw == "" {
return time.Time{}
}
parsed, err := time.Parse(time.RFC3339Nano, raw)
if err != nil {
return time.Time{}
}
return parsed
}
func pluginCategoryTrailReferences(app core.App, userID string, pluginID string) ([]pluginCategoryTrailReference, error) {
if userID == "" || pluginID == "" {
return nil, nil
}
refs, err := app.FindRecordsByFilter(
"trail_external_reference",
"user={:user} && plugin_id={:plugin_id}",
"",
-1,
0,
dbx.Params{"user": userID, "plugin_id": pluginID},
)
if err != nil || len(refs) == 0 {
return nil, err
}
trailIDs := make([]string, 0, len(refs))
seen := map[string]bool{}
for _, ref := range refs {
trailID := ref.GetString("trail")
if trailID == "" || seen[trailID] {
continue
}
seen[trailID] = true
trailIDs = append(trailIDs, trailID)
}
if len(trailIDs) == 0 {
return nil, nil
}
trails, err := app.FindRecordsByIds("trails", trailIDs)
if err != nil {
return nil, err
}
trailsByID := make(map[string]*core.Record, len(trails))
for _, trail := range trails {
trailsByID[trail.Id] = trail
}
result := make([]pluginCategoryTrailReference, 0, len(trails))
seen = map[string]bool{}
for _, ref := range refs {
trailID := ref.GetString("trail")
if trailID == "" || seen[trailID] {
continue
}
trail := trailsByID[trailID]
if trail == nil {
continue
}
seen[trailID] = true
result = append(result, pluginCategoryTrailReference{
Ref: ref,
Trail: trail,
ExternalID: ref.GetString("external_id"),
})
}
return result, nil
}

42
db/routes/plugin_system_config.go Normal file
View File

@@ -0,0 +1,42 @@
package routes
import (
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"pocketbase/pluginsystem"
)
func effectivePluginConfig(app core.App, pluginID string, instance *core.Record) map[string]any {
config := installedPluginConfig(app, pluginID)
pluginsystem.MergePluginConfig(config, pluginsystem.JSONMapFromRecord(instance, "config"))
return config
}
func pluginRuntimeConfig(config map[string]any) map[string]any {
return configSection(config, "plugin")
}
func pluginHostConfig(config map[string]any) map[string]any {
return configSection(config, "host")
}
func configSection(config map[string]any, key string) map[string]any {
raw, ok := config[key].(map[string]any)
if !ok || raw == nil {
return map[string]any{}
}
return raw
}
func installedPluginConfig(app core.App, pluginID string) map[string]any {
record, _ := app.FindFirstRecordByFilter(
"installed_plugins",
"plugin_id={:plugin_id}",
dbx.Params{"plugin_id": pluginID},
)
if record == nil {
return map[string]any{}
}
return pluginsystem.JSONMapFromRecord(record, "config")
}

147
db/routes/plugin_system_policy.go Normal file
View File

@@ -0,0 +1,147 @@
package routes
import (
"encoding/base64"
"fmt"
"strings"
"pocketbase/pluginsystem"
)
func pluginInstancePolicy(plugin pluginsystem.LocalPlugin, config map[string]any) pluginsystem.RequestPolicyContext {
connectors := map[string]pluginsystem.ResolvedConnectorTarget{}
hostConfig := pluginHostConfig(config)
hostConnectors := configMap(configMap(hostConfig, "connectors"), "")
for _, manifestConnector := range plugin.Manifest.Permissions.Network.Connectors {
target, err := resolveConnectorTarget(manifestConnector, hostConnectors)
if err != nil {
continue
}
connectors[manifestConnector.Name] = target
}
return pluginsystem.RequestPolicyContext{Connectors: connectors}
}
func resolveConnectorTarget(manifest pluginsystem.ConnectorTargetPermission, hostConnectors map[string]any) (pluginsystem.ResolvedConnectorTarget, error) {
target := pluginsystem.ResolvedConnectorTarget{
Name: manifest.Name,
Type: manifest.Type,
AllowedPathPrefixes: manifest.AllowedPathPrefixes,
Auth: manifest.Auth,
SupportsMediaAuth: manifest.SupportsMediaAuth,
SupportsStorageRedirects: manifest.SupportsStorageRedirects,
SupportsCustomTLS: manifest.SupportsCustomTLS,
TLS: pluginsystem.ConnectorTLSConfig{Mode: pluginsystem.TLSModeSystem},
StorageOrigins: map[string]pluginsystem.ResolvedConnectorOrigin{},
}
switch manifest.Type {
case pluginsystem.ConnectorTypePublicAPI:
baseURL, basePath, err := pluginsystem.NormalizeConnectorBase(manifest.FixedBaseURL, "")
if err != nil {
return target, err
}
target.BaseURL = baseURL
target.BasePath = basePath
target.AllowPrivate = false
case pluginsystem.ConnectorTypeConfigured:
rawConfig := configMap(hostConnectors, manifest.ConfigKey)
if len(rawConfig) == 0 {
return target, fmt.Errorf("configured connector %q has no host config", manifest.Name)
}
baseURL := stringConfig(rawConfig, "baseURL")
basePath := stringConfig(rawConfig, "basePath")
normalizedBaseURL, normalizedBasePath, err := pluginsystem.NormalizeConnectorBase(baseURL, basePath)
if err != nil {
return target, err
}
target.BaseURL = normalizedBaseURL
target.BasePath = normalizedBasePath
target.AllowPrivate = boolConfig(rawConfig, "allowPrivate")
target.TLS = tlsConfig(rawConfig, manifest.SupportsCustomTLS)
if manifest.SupportsStorageRedirects {
target.StorageOrigins = storageOrigins(rawConfig)
}
default:
return target, fmt.Errorf("unsupported connector type %q", manifest.Type)
}
return target, nil
}
func storageOrigins(rawConfig map[string]any) map[string]pluginsystem.ResolvedConnectorOrigin {
rawOrigins := configMap(rawConfig, "storageOrigins")
origins := map[string]pluginsystem.ResolvedConnectorOrigin{}
for name, raw := range rawOrigins {
originMap, ok := raw.(map[string]any)
if !ok {
continue
}
baseURL, basePath, err := pluginsystem.NormalizeConnectorBase(
stringConfig(originMap, "baseURL"),
stringConfig(originMap, "basePath"),
)
if err != nil {
continue
}
origins[name] = pluginsystem.ResolvedConnectorOrigin{
Name: name,
BaseURL: baseURL,
BasePath: basePath,
AllowPrivate: boolConfig(originMap, "allowPrivate"),
TLS: tlsConfig(originMap, true),
}
}
return origins
}
func tlsConfig(raw map[string]any, customAllowed bool) pluginsystem.ConnectorTLSConfig {
rawTLS := configMap(raw, "tls")
mode := stringConfig(rawTLS, "mode")
if mode == "" {
mode = pluginsystem.TLSModeSystem
}
if mode != pluginsystem.TLSModeSystem && mode != pluginsystem.TLSModeCustomCA {
mode = pluginsystem.TLSModeSystem
}
if !customAllowed && mode != pluginsystem.TLSModeSystem {
mode = pluginsystem.TLSModeSystem
}
cfg := pluginsystem.ConnectorTLSConfig{Mode: mode}
if mode == pluginsystem.TLSModeCustomCA {
ca := stringConfig(rawTLS, "caBundle")
if decoded, err := base64.StdEncoding.DecodeString(ca); err == nil {
cfg.CABundle = decoded
} else {
cfg.CABundle = []byte(ca)
}
}
return cfg
}
func configMap(raw map[string]any, key string) map[string]any {
if key == "" {
return raw
}
value, ok := raw[key]
if !ok {
return map[string]any{}
}
switch typed := value.(type) {
case map[string]any:
return typed
default:
return map[string]any{}
}
}
func stringConfig(raw map[string]any, key string) string {
value, _ := raw[key].(string)
return strings.TrimSpace(value)
}
func boolConfig(raw map[string]any, key string) bool {
value, _ := raw[key].(bool)
return value
}

55
db/routes/plugin_system_policy_test.go Normal file
View File

@@ -0,0 +1,55 @@
package routes
import (
"testing"
"pocketbase/pluginsystem"
)
func TestPluginInstancePolicyUsesHostConnectorConfig(t *testing.T) {
plugin := pluginsystem.LocalPlugin{Manifest: pluginsystem.Manifest{
Permissions: pluginsystem.PermissionManifest{
Network: pluginsystem.NetworkPermissions{
Connectors: []pluginsystem.ConnectorTargetPermission{{
Name: "media",
Type: pluginsystem.ConnectorTypeConfigured,
ConfigKey: "immich",
SupportsCustomTLS: true,
}},
},
},
}}
config := map[string]any{
"plugin": map[string]any{
"after": "2026-01-01",
},
"host": map[string]any{
"connectors": map[string]any{
"immich": map[string]any{
"baseURL": "https://photos.example.test",
"basePath": "/immich",
"allowPrivate": true,
"tls": map[string]any{
"mode": pluginsystem.TLSModeCustomCA,
"caBundle": "test-ca",
},
},
},
},
}
policy := pluginInstancePolicy(plugin, config)
connector, ok := policy.Connectors["media"]
if !ok {
t.Fatal("expected configured connector to be resolved from host config")
}
if connector.BaseURL != "https://photos.example.test" || connector.BasePath != "/immich" {
t.Fatalf("unexpected connector base: %#v", connector)
}
if !connector.AllowPrivate {
t.Fatal("expected allowPrivate from host connector config")
}
if connector.TLS.Mode != pluginsystem.TLSModeCustomCA || string(connector.TLS.CABundle) != "test-ca" {
t.Fatalf("unexpected TLS config: %#v", connector.TLS)
}
}

223
db/routes/plugin_system_send.go Normal file
View File

@@ -0,0 +1,223 @@
package routes
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"strings"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"github.com/pocketbase/pocketbase/tools/security"
"pocketbase/pluginsystem"
"pocketbase/util"
)
type pluginSystemTrailSendRequest struct {
PluginID string `json:"pluginId"`
TrailID string `json:"trailId"`
Share string `json:"share,omitempty"`
}
type pluginSystemTrailSendInput struct {
Instance pluginsystem.InstanceRef `json:"instance"`
Auth map[string]any `json:"auth,omitempty"`
Config map[string]any `json:"config,omitempty"`
Name string `json:"name,omitempty"`
Trail pluginsystem.Track `json:"trail"`
}
// PluginSystemTrailSend asks a plugin to prepare a trail send request for an
// existing trail and then executes that request through the host policy layer.
func PluginSystemTrailSend(e *core.RequestEvent) error {
if e.Auth == nil {
return apis.NewUnauthorizedError("authentication required", nil)
}
var data pluginSystemTrailSendRequest
if err := e.BindBody(&data); err != nil {
return apis.NewBadRequestError("Failed to read request data", err)
}
if data.PluginID == "" || data.TrailID == "" {
return apis.NewBadRequestError("pluginId and trailId are required", nil)
}
instance, err := e.App.FindFirstRecordByFilter(
"plugin_instances",
"user={:user} && plugin_id={:plugin_id} && enabled=true",
dbx.Params{"user": e.Auth.Id, "plugin_id": data.PluginID},
)
if err != nil {
return apis.NewBadRequestError("no enabled plugin instance configured for this plugin", nil)
}
plugin, capability, err := localPluginCapability(e.App, data.PluginID, "prepare_trail_send", "v1")
if err != nil {
return err
}
trail, err := e.App.FindRecordById("trails", data.TrailID)
if err != nil {
return apis.NewNotFoundError("trail not found", nil)
}
if !util.TrailViewableByUser(e.App, trail, e.Auth.Id, data.Share) {
return apis.NewForbiddenError("not allowed to send this trail", nil)
}
gpx, err := readTrailGPX(e.App, trail)
if err != nil {
return err
}
if len(gpx) == 0 {
return apis.NewBadRequestError("trail has no GPX track", nil)
}
auth, err := decryptedInstanceAuth(instance)
if err != nil {
return err
}
input := pluginSystemTrailSendInput{
Instance: pluginsystem.InstanceRef{
ID: instance.Id,
PluginID: instance.GetString("plugin_id"),
},
Auth: pluginsystem.PluginInputAuth(plugin, auth),
Name: trail.GetString("name"),
Trail: pluginsystem.Track{
Format: "gpx",
ContentBase64: base64.StdEncoding.EncodeToString(gpx),
},
}
config := effectivePluginConfig(e.App, plugin.Manifest.ID, instance)
pluginConfig := pluginRuntimeConfig(config)
policy := pluginInstancePolicy(plugin, config)
input.Config = pluginConfig
inputBytes, err := json.Marshal(input)
if err != nil {
return err
}
runtime, err := pluginsystem.NewRuntimeRegistry().RuntimeFor(plugin)
if err != nil {
return err
}
session, err := runtime.OpenSession(e.Request.Context(), plugin, policy.WithHostAuth(auth))
if err != nil {
return err
}
defer func() {
_ = session.Close(context.Background())
}()
output, err := session.Call(e.Request.Context(), capability.Export, inputBytes)
if err != nil {
return err
}
var plan pluginsystem.TrailSendPlan
if err := json.Unmarshal(output, &plan); err != nil {
return apis.NewBadRequestError("plugin returned an invalid send plan", err)
}
if plan.Request.Method == "" {
return apis.NewBadRequestError("plugin returned an empty send request", nil)
}
if err := pluginsystem.ValidateHostRequestSpec(plugin.Manifest, plan.Request, policy); err != nil {
return apis.NewBadRequestError("plugin send request is not permitted by manifest", err)
}
if err := pluginsystem.InjectHostRequestAuth(e.Request.Context(), pluginsystem.AuthInjectionInput{
App: e.App,
Runtime: runtime,
Session: session,
Plugin: plugin,
Instance: instance,
Auth: auth,
Config: pluginConfig,
Spec: &plan.Request,
Policy: policy,
}); err != nil {
return apis.NewBadRequestError("plugin auth injection failed", err)
}
// Auth is fully resolved above (including OAuth refresh and plugin session
// refresh). Clearing the reference makes this handler the sole injector so the
// executor's policy-based injection becomes a no-op instead of re-injecting
// against an empty policy.HostAuth.
plan.Request.Auth = ""
if err := executeHostRequest(e.Request.Context(), plugin.Manifest, policy, plan.Request, gpx); err != nil {
return err
}
return e.JSON(http.StatusOK, map[string]any{"ok": true})
}
// executeHostRequest runs a plugin send plan through the shared host request
// executor and maps provider failures to API errors.
func executeHostRequest(ctx context.Context, manifest pluginsystem.Manifest, policy pluginsystem.RequestPolicyContext, spec pluginsystem.HostRequestSpec, gpx []byte) error {
resp, err := pluginsystem.ExecuteHostRequest(ctx, manifest, policy, spec, pluginsystem.HostRequestOptions{
Trail: gpx,
})
if err != nil {
return err
}
if resp.Status < 200 || resp.Status >= 300 {
return apis.NewBadRequestError(
fmt.Sprintf("provider request failed: %d", resp.Status),
strings.TrimSpace(string(resp.Body)),
)
}
return nil
}
// readTrailGPX loads the trail GPX file that can be inserted into a plugin's
// multipart send plan.
func readTrailGPX(app core.App, trail *core.Record) ([]byte, error) {
gpxPath := trail.GetString("gpx")
if gpxPath == "" {
return nil, nil
}
fsys, err := app.NewFilesystem()
if err != nil {
return nil, err
}
defer fsys.Close()
reader, err := fsys.GetReader(trail.BaseFilesPath() + "/" + gpxPath)
if err != nil {
return nil, err
}
defer reader.Close()
return io.ReadAll(reader)
}
// decryptedInstanceAuth returns auth fields in the shape expected by host-side
// auth injection and plugin input preparation.
func decryptedInstanceAuth(instance *core.Record) (map[string]any, error) {
auth := pluginsystem.JSONMapFromRecord(instance, "auth")
if len(auth) == 0 {
return map[string]any{}, nil
}
encryptionKey := os.Getenv("POCKETBASE_ENCRYPTION_KEY")
if encryptionKey == "" {
return nil, apis.NewBadRequestError("POCKETBASE_ENCRYPTION_KEY not set", nil)
}
for key, value := range auth {
secret, ok := value.(string)
if !ok || secret == "" || !util.CanDecryptSecret(secret) {
continue
}
decrypted, err := security.Decrypt(secret, encryptionKey)
if err != nil {
return nil, fmt.Errorf("decrypt %s: %w", key, err)
}
auth[key] = string(decrypted)
}
return auth, nil
}

119
db/routes/plugin_system_session_auth.go Normal file
View File

@@ -0,0 +1,119 @@
package routes
import (
"encoding/json"
"net/http"
"github.com/pocketbase/pocketbase/apis"
"github.com/pocketbase/pocketbase/core"
"pocketbase/pluginsystem"
)
type pluginSessionAuthValidateRequest struct {
PluginID string `json:"pluginId"`
InstanceID string `json:"instanceId,omitempty"`
AuthContext string `json:"authContext,omitempty"`
Auth map[string]any `json:"auth,omitempty"`
}
type pluginSessionAuthRefreshInput struct {
Instance pluginsystem.InstanceRef `json:"instance"`
Auth map[string]any `json:"auth,omitempty"`
}
func PluginSystemSessionAuthValidate(e *core.RequestEvent) error {
if e.Auth == nil {
return apis.NewUnauthorizedError("authentication required", nil)
}
var data pluginSessionAuthValidateRequest
if err := e.BindBody(&data); err != nil {
return apis.NewBadRequestError("failed to read request data", err)
}
if data.PluginID == "" {
return apis.NewBadRequestError("pluginId is required", nil)
}
plugin, err := localPlugin(e.App, data.PluginID)
if err != nil {
return err
}
contextName, authContext, err := sessionAuthContext(plugin, data.AuthContext)
if err != nil {
return apis.NewBadRequestError("plugin has no session auth context", err)
}
if authContext.Refresh == nil || authContext.Refresh.Function == "" {
return apis.NewBadRequestError("plugin session auth context has no refresh function", nil)
}
auth := map[string]any{}
instanceID := data.InstanceID
if instanceID != "" {
instance, err := pluginAuthInstance(e.App, e.Auth.Id, data.PluginID, instanceID)
if err != nil {
return err
}
instanceID = instance.Id
auth, err = decryptedInstanceAuth(instance)
if err != nil {
return err
}
}
for key, value := range data.Auth {
if value == "" {
continue
}
auth[key] = value
}
inputBytes, err := json.Marshal(pluginSessionAuthRefreshInput{
Instance: pluginsystem.InstanceRef{
ID: instanceID,
PluginID: plugin.Manifest.ID,
},
Auth: pluginsystem.AuthForPluginRefresh(auth, authContext),
})
if err != nil {
return err
}
runtime, err := pluginsystem.NewRuntimeRegistry().RuntimeFor(plugin)
if err != nil {
return err
}
// TODO: accept and merge plugin instance config here before supporting
// session-auth plugins with configured connectors. The current validation
// path is sufficient for public_api session plugins such as komoot and
// hammerhead, but configured connectors need host config for policy
// resolution and refresh input parity with production auth injection.
policy := pluginInstancePolicy(plugin, map[string]any{}).WithHostAuth(auth)
output, err := runtime.Call(e.Request.Context(), plugin, authContext.Refresh.Function, inputBytes, policy)
if err != nil {
return apis.NewBadRequestError("plugin credentials validation failed", err)
}
if err := pluginsystem.ValidatePluginSessionRefreshOutput(output); err != nil {
return apis.NewBadRequestError("plugin credentials validation failed", err)
}
return e.JSON(http.StatusOK, map[string]any{
"ok": true,
"authContext": contextName,
})
}
func sessionAuthContext(plugin pluginsystem.LocalPlugin, requested string) (string, pluginsystem.AuthContext, error) {
if requested != "" {
authContext, ok := plugin.Manifest.Auth.Contexts[requested]
if !ok || authContext.Type != pluginsystem.AuthTypeSession {
return "", pluginsystem.AuthContext{}, apis.NewBadRequestError("unknown session auth context", nil)
}
return requested, authContext, nil
}
for name, authContext := range plugin.Manifest.Auth.Contexts {
if authContext.Type == pluginsystem.AuthTypeSession {
return name, authContext, nil
}
}
return "", pluginsystem.AuthContext{}, apis.NewBadRequestError("session auth context not found", nil)
}

619
db/routes/plugin_system_sync.go Normal file
View File

@@ -0,0 +1,619 @@
package routes
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
"pocketbase/plugins/importer"
"pocketbase/pluginsystem"
"pocketbase/services/trailmerge"
"pocketbase/util"
)
const (
defaultPluginSyncBatchLimit = 50
defaultPluginSyncMaxBatches = 100
defaultPluginProviderCategoryBackfillLimit = 10
)
var syncCapabilityDescriptors = []syncCapabilityDescriptor{
{
OptionKey: "planned",
CapabilityName: "list_routes",
DetailName: "get_route_detail",
Version: "v1",
},
{
OptionKey: "completed",
CapabilityName: "list_activities",
DetailName: "get_activity_detail",
Version: "v1",
},
}
type syncCapabilityDescriptor struct {
OptionKey string
CapabilityName string
DetailName string
Version string
}
type pluginSystemListInput struct {
Instance pluginsystem.InstanceRef `json:"instance"`
Auth map[string]any `json:"auth,omitempty"`
State map[string]any `json:"state,omitempty"`
Options map[string]any `json:"options,omitempty"`
Limits pluginSystemSyncLimits `json:"limits,omitempty"`
}
type pluginSystemSyncLimits struct {
MaxItems int `json:"maxItems,omitempty"`
}
type pluginSystemListOutput struct {
Items []pluginsystem.TrailSummary `json:"items"`
State map[string]any `json:"state,omitempty"`
HasMore bool `json:"hasMore"`
Error *pluginsystem.PluginError `json:"error,omitempty"`
}
type pluginSystemDetailInput struct {
Instance pluginsystem.InstanceRef `json:"instance"`
Auth map[string]any `json:"auth,omitempty"`
Options map[string]any `json:"options,omitempty"`
Summary pluginsystem.TrailSummary `json:"summary"`
}
type pluginSystemDetailOutput struct {
Item pluginsystem.TrailImport `json:"item"`
Error *pluginsystem.PluginError `json:"error,omitempty"`
}
type pluginSystemSyncResult struct {
PluginID string `json:"pluginId"`
Imported int `json:"imported"`
Skipped int `json:"skipped"`
}
// PluginSystemSyncConfigured is the cron entrypoint. It refreshes plugin
// metadata, finds enabled instances, skips instances in backoff, and syncs each
// configured import capability.
func PluginSystemSyncConfigured(ctx context.Context, app core.App, client meilisearch.ServiceManager) error {
app.Logger().Info("plugin sync cron started")
manager := pluginsystem.NewManager(app, "")
if err := manager.SyncInstalledPlugins(ctx); err != nil {
return err
}
plugins, err := pluginsystem.LoadInstalledPlugins(app, "")
if err != nil {
return err
}
app.Logger().Info("plugin sync discovered installed plugins", "count", len(plugins))
var syncErr error
for _, plugin := range plugins {
if !pluginHasAnySyncCapability(plugin) {
app.Logger().Info("plugin sync skipping plugin without sync capability", "plugin", plugin.Manifest.ID)
continue
}
instances, err := pluginInstances(app, plugin.Manifest.ID)
if err != nil {
return err
}
app.Logger().Info("plugin sync found enabled instances", "plugin", plugin.Manifest.ID, "count", len(instances))
for _, instance := range instances {
if err := ctx.Err(); err != nil {
return err
}
if shouldSkipPluginInstance(instance) {
app.Logger().Info("plugin sync skipping instance due to retry delay", "plugin", plugin.Manifest.ID, "instance", instance.Id, "retry_not_before", instance.GetString("retry_not_before"))
continue
}
app.Logger().Info("plugin instance sync started", "plugin", plugin.Manifest.ID, "instance", instance.Id)
result, err := syncPluginInstance(ctx, app, client, plugin, instance)
if err != nil {
app.Logger().Warn("plugin instance sync failed", "plugin", plugin.Manifest.ID, "instance", instance.Id, "error", err)
syncErr = err
continue
}
app.Logger().Info("plugin instance sync completed", "plugin", result.PluginID, "instance", instance.Id, "imported", result.Imported, "skipped", result.Skipped)
}
}
app.Logger().Info("plugin sync cron completed")
return syncErr
}
func pluginInstances(app core.App, pluginID string) ([]*core.Record, error) {
return app.FindRecordsByFilter(
"plugin_instances",
"plugin_id={:plugin_id} && enabled=true",
"",
-1,
0,
dbx.Params{"plugin_id": pluginID},
)
}
// syncPluginInstance prepares one plugin instance for import: it resolves the
// actor, creates the runtime, decrypts/refreshes auth, and dispatches every
// enabled sync capability.
func syncPluginInstance(ctx context.Context, app core.App, client meilisearch.ServiceManager, plugin pluginsystem.LocalPlugin, instance *core.Record) (*pluginSystemSyncResult, error) {
actor, err := app.FindFirstRecordByData("activitypub_actors", "user", instance.GetString("user"))
if err != nil {
setPluginInstanceStatus(app, instance, "error", "invalid_request", "activitypub actor not found")
return nil, err
}
auth, err := decryptedInstanceAuth(instance)
if err != nil {
setPluginInstanceStatus(app, instance, "needs_reauth", "auth_failed", err.Error())
return nil, err
}
auth, err = pluginsystem.RefreshOAuthAuthIfNeeded(ctx, app, plugin, instance, auth)
if err != nil {
setPluginInstanceStatus(app, instance, "needs_reauth", "auth_failed", err.Error())
return nil, err
}
config := effectivePluginConfig(app, plugin.Manifest.ID, instance)
pluginConfig := pluginRuntimeConfig(config)
hostConfig := pluginHostConfig(config)
defaultPublic := userDefaultPublic(app, instance.GetString("user"))
createSummitLog := boolOption(hostConfig, "createSummitLogForCompleted", true)
runtime, err := pluginsystem.NewRuntimeRegistry().RuntimeFor(plugin)
if err != nil {
setPluginInstanceStatusForError(app, instance, err)
return nil, err
}
sessions := &pluginSyncRuntimeSession{
runtime: runtime,
plugin: plugin,
policy: pluginInstancePolicy(plugin, config).WithHostAuth(auth),
}
if err := sessions.open(ctx); err != nil {
setPluginInstanceStatusForError(app, instance, err)
return nil, err
}
defer func() {
_ = sessions.close(context.Background())
}()
instance.Set("status", "syncing")
if err := app.Save(instance); err != nil {
return nil, err
}
result := &pluginSystemSyncResult{PluginID: plugin.Manifest.ID}
for _, descriptor := range syncCapabilityDescriptors {
if !boolOption(hostConfig, descriptor.OptionKey, true) {
app.Logger().Info("plugin sync skipping disabled capability", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", descriptor.CapabilityName, "option", descriptor.OptionKey)
continue
}
if !pluginHasCapability(plugin, descriptor.CapabilityName, descriptor.Version) {
app.Logger().Info("plugin sync skipping unavailable capability", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", descriptor.CapabilityName, "version", descriptor.Version)
continue
}
if !pluginHasCapability(plugin, descriptor.DetailName, descriptor.Version) {
app.Logger().Warn("plugin sync skipping list capability because matching detail capability is unavailable", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", descriptor.CapabilityName, "detail_capability", descriptor.DetailName, "version", descriptor.Version)
continue
}
capability, err := pluginCapability(plugin, descriptor.CapabilityName, descriptor.Version)
if err != nil {
return nil, err
}
detailCapability, err := pluginCapability(plugin, descriptor.DetailName, descriptor.Version)
if err != nil {
return nil, err
}
app.Logger().Info("plugin capability sync started", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", capability.Name, "version", capability.Version, "export", capability.Export)
capResult, err := syncPluginCapability(ctx, app, client, sessions, plugin, capability, detailCapability, instance, actor, auth, pluginConfig, hostConfig, defaultPublic, createSummitLog)
if err != nil {
setPluginInstanceStatusForError(app, instance, err)
return nil, err
}
app.Logger().Info("plugin capability sync completed", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", capability.Name, "imported", capResult.Imported, "skipped", capResult.Skipped)
result.Imported += capResult.Imported
result.Skipped += capResult.Skipped
}
instance.Set("state", map[string]any{})
instance.Set("last_sync_at", time.Now())
instance.Set("last_error", map[string]any{})
instance.Set("retry_not_before", "")
instance.Set("status", "configured")
if err := app.Save(instance); err != nil {
return nil, err
}
return result, nil
}
// shouldSkipPluginInstance applies retry delay from the last sync error.
func shouldSkipPluginInstance(instance *core.Record) bool {
retryNotBefore := instance.GetDateTime("retry_not_before")
return !retryNotBefore.IsZero() && retryNotBefore.Time().After(time.Now())
}
type capabilitySyncResult struct {
Imported int
Skipped int
}
type pluginSyncRuntimeSession struct {
runtime pluginsystem.Runtime
plugin pluginsystem.LocalPlugin
policy pluginsystem.RequestPolicyContext
session pluginsystem.RuntimeSession
}
func (s *pluginSyncRuntimeSession) open(ctx context.Context) error {
session, err := s.runtime.OpenSession(ctx, s.plugin, s.policy)
if err != nil {
return err
}
s.session = session
return nil
}
func (s *pluginSyncRuntimeSession) reopen(ctx context.Context) error {
_ = s.close(context.Background())
return s.open(ctx)
}
func (s *pluginSyncRuntimeSession) close(ctx context.Context) error {
if s.session == nil {
return nil
}
err := s.session.Close(ctx)
s.session = nil
return err
}
// syncPluginCapability calls one plugin export such as list_routes_v1, imports
// the returned trail items, and carries transient page state only within this
// sync run. The page cursor is intentionally not persisted across runs.
func syncPluginCapability(ctx context.Context, app core.App, client meilisearch.ServiceManager, sessions *pluginSyncRuntimeSession, plugin pluginsystem.LocalPlugin, capability pluginsystem.CapabilityManifest, detailCapability pluginsystem.CapabilityManifest, instance *core.Record, actor *core.Record, auth map[string]any, pluginConfig map[string]any, hostConfig map[string]any, defaultPublic bool, createSummitLog bool) (*capabilitySyncResult, error) {
result := &capabilitySyncResult{}
state := map[string]any{}
hasMore := true
policy := sessions.policy
providerCategoryBackfillsRemaining := 0
if hasUsableCategoryMapping(categoryMapping(hostConfig)) {
providerCategoryBackfillsRemaining = defaultPluginProviderCategoryBackfillLimit
}
for batch := 0; hasMore && batch < defaultPluginSyncMaxBatches; batch++ {
input := pluginSystemListInput{
Instance: pluginsystem.InstanceRef{
ID: instance.Id,
PluginID: instance.GetString("plugin_id"),
},
Auth: pluginsystem.PluginInputAuth(plugin, auth),
State: state,
Options: pluginConfig,
Limits: pluginSystemSyncLimits{MaxItems: defaultPluginSyncBatchLimit},
}
inputBytes, err := json.Marshal(input)
if err != nil {
return nil, err
}
outputBytes, err := sessions.session.Call(ctx, capability.Export, inputBytes)
if err != nil {
return nil, err
}
var output pluginSystemListOutput
if err := json.Unmarshal(outputBytes, &output); err != nil {
return nil, fmt.Errorf("plugin returned invalid %s output: %w", capability.Export, err)
}
if output.Error != nil {
return nil, pluginsystem.PluginCapabilityError{Err: output.Error}
}
app.Logger().Info("plugin capability batch returned items", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", capability.Name, "batch", batch, "items", len(output.Items), "has_more", output.HasMore)
summaries := output.Items
externalIDsByProvider := map[string][]string{}
for i := range summaries {
if summaries[i].Source.Provider == "" {
summaries[i].Source.Provider = plugin.Manifest.ID
}
if summaries[i].Source.ExternalID == "" {
continue
}
externalIDsByProvider[summaries[i].Source.Provider] = append(externalIDsByProvider[summaries[i].Source.Provider], summaries[i].Source.ExternalID)
}
existingIDsByProvider := map[string]map[string]bool{}
providerCategoryBackfillCandidatesByProvider := map[string]map[string]*core.Record{}
for provider, externalIDs := range externalIDsByProvider {
existingIDs, err := util.FindExistingExternalReferenceIDsForUser(app, instance.GetString("user"), provider, externalIDs)
if err != nil {
return nil, err
}
existingIDsByProvider[provider] = existingIDs
if providerCategoryBackfillsRemaining > 0 && len(existingIDs) > 0 {
candidates, err := providerCategoryBackfillCandidatesForSync(app, instance.GetString("user"), provider, externalIDs, providerCategoryBackfillsRemaining)
if err != nil {
return nil, err
}
providerCategoryBackfillCandidatesByProvider[provider] = candidates
}
}
for _, summary := range summaries {
if summary.Source.ExternalID == "" {
continue
}
if existingIDsByProvider[summary.Source.Provider][summary.Source.ExternalID] {
result.Skipped++
if providerCategoryBackfillsRemaining > 0 {
ref := providerCategoryBackfillCandidatesByProvider[summary.Source.Provider][summary.Source.ExternalID]
attempted, err := backfillProviderCategoryDuringSync(ctx, app, sessions, plugin, detailCapability, instance, auth, pluginConfig, summary, ref)
if err != nil {
return nil, err
}
if attempted {
providerCategoryBackfillsRemaining--
}
}
continue
}
item, err := pluginDetail(ctx, sessions.session, plugin, detailCapability, instance, auth, pluginConfig, summary)
if err != nil {
result.Skipped++
app.Logger().Warn("skipping plugin item after detail fetch failed", "plugin", plugin.Manifest.ID, "instance", instance.Id, "capability", capability.Name, "provider", summary.Source.Provider, "external_id", summary.Source.ExternalID, "error", err)
if pluginsystem.IsRuntimeSessionFatalError(err) {
if reopenErr := sessions.reopen(ctx); reopenErr != nil {
return nil, reopenErr
}
}
continue
}
applyHostPolicy(&item, hostConfig)
imported, err := importer.ImportTrail(ctx, app, item, importer.Options{
UserID: instance.GetString("user"),
ActorID: actor.Id,
DefaultPublic: defaultPublic,
CreateSummitLogForCompleted: createSummitLog,
CategoryMapping: categoryMapping(hostConfig),
Manifest: plugin.Manifest,
Policy: policy,
Auth: auth,
})
if err != nil {
return nil, err
}
if imported.Created {
result.Imported++
app.Logger().Info("imported plugin trail", "provider", item.Source.Provider, "external_id", item.Source.ExternalID, "trail", imported.TrailID)
if autoMergeEnabled(hostConfig) {
settings := trailmerge.DefaultPluginAutoMergeSettings()
settings.Enabled = true
if err := trailmerge.TryAutoMergeImportedTrail(app, client, ctx, actor, imported.TrailID, settings); err != nil {
app.Logger().Warn("unable to auto-merge imported plugin trail", "provider", item.Source.Provider, "external_id", item.Source.ExternalID, "trail", imported.TrailID, "error", err)
}
}
}
if imported.Skipped {
result.Skipped++
}
}
state = output.State
if state == nil {
state = map[string]any{}
}
hasMore = output.HasMore
}
if hasMore {
return nil, fmt.Errorf("sync stopped after %d batches", defaultPluginSyncMaxBatches)
}
return result, nil
}
func providerCategoryBackfillCandidatesForSync(app core.App, userID string, provider string, externalIDs []string, limit int) (map[string]*core.Record, error) {
candidates := map[string]*core.Record{}
if userID == "" || provider == "" || len(externalIDs) == 0 || limit <= 0 {
return candidates, nil
}
params := dbx.Params{
"user": userID,
"provider": provider,
}
seenExternalIDs := map[string]bool{}
idFilters := make([]string, 0, len(externalIDs))
for _, externalID := range externalIDs {
if externalID == "" || seenExternalIDs[externalID] {
continue
}
seenExternalIDs[externalID] = true
paramName := fmt.Sprintf("external_id_%d", len(idFilters))
params[paramName] = externalID
idFilters = append(idFilters, "external_id={:"+paramName+"}")
}
if len(idFilters) == 0 {
return candidates, nil
}
filter := "user={:user} && provider={:provider} && (" + strings.Join(idFilters, " || ") + ")"
refs, err := app.FindRecordsByFilter("trail_external_reference", filter, "", len(idFilters), 0, params)
if err != nil || len(refs) == 0 {
return candidates, err
}
for _, ref := range refs {
if len(candidates) >= limit {
break
}
if ref.GetString("provider_category") != "" || !ref.GetDateTime("provider_category_checked_at").IsZero() {
continue
}
candidates[ref.GetString("external_id")] = ref
}
return candidates, nil
}
func backfillProviderCategoryDuringSync(ctx context.Context, app core.App, sessions *pluginSyncRuntimeSession, plugin pluginsystem.LocalPlugin, detailCapability pluginsystem.CapabilityManifest, instance *core.Record, auth map[string]any, pluginConfig map[string]any, summary pluginsystem.TrailSummary, ref *core.Record) (bool, error) {
if ref == nil {
return false, nil
}
item, err := pluginDetail(ctx, sessions.session, plugin, detailCapability, instance, auth, pluginConfig, summary)
if err != nil {
app.Logger().Warn("skipping provider category backfill after detail fetch failed", "plugin", plugin.Manifest.ID, "instance", instance.Id, "provider", summary.Source.Provider, "external_id", summary.Source.ExternalID, "error", err)
if pluginsystem.IsRuntimeSessionFatalError(err) {
if reopenErr := sessions.reopen(ctx); reopenErr != nil {
return true, reopenErr
}
}
return true, nil
}
ref.Set("provider_category", importer.ProviderCategoryFromImport(item))
ref.Set("provider_category_checked_at", time.Now())
if err := app.Save(ref); err != nil {
return false, err
}
return true, nil
}
func pluginDetail(ctx context.Context, session pluginsystem.RuntimeSession, plugin pluginsystem.LocalPlugin, capability pluginsystem.CapabilityManifest, instance *core.Record, auth map[string]any, pluginConfig map[string]any, summary pluginsystem.TrailSummary) (pluginsystem.TrailImport, error) {
input := pluginSystemDetailInput{
Instance: pluginsystem.InstanceRef{
ID: instance.Id,
PluginID: instance.GetString("plugin_id"),
},
Auth: pluginsystem.PluginInputAuth(plugin, auth),
Options: pluginConfig,
Summary: summary,
}
inputBytes, err := json.Marshal(input)
if err != nil {
return pluginsystem.TrailImport{}, err
}
outputBytes, err := session.Call(ctx, capability.Export, inputBytes)
if err != nil {
return pluginsystem.TrailImport{}, err
}
var output pluginSystemDetailOutput
if err := json.Unmarshal(outputBytes, &output); err != nil {
return pluginsystem.TrailImport{}, fmt.Errorf("plugin returned invalid %s output: %w", capability.Export, err)
}
if output.Error != nil {
return pluginsystem.TrailImport{}, pluginsystem.PluginCapabilityError{Err: output.Error}
}
return output.Item, nil
}
func pluginHasCapability(plugin pluginsystem.LocalPlugin, name string, version string) bool {
for _, capability := range plugin.Manifest.Capabilities {
if capability.Name == name && capability.Version == version {
return true
}
}
return false
}
func pluginHasAnySyncCapability(plugin pluginsystem.LocalPlugin) bool {
for _, descriptor := range syncCapabilityDescriptors {
if pluginHasCapability(plugin, descriptor.CapabilityName, descriptor.Version) {
return true
}
}
return false
}
func setPluginInstanceStatus(app core.App, instance *core.Record, status string, code string, message string) {
instance.Set("status", status)
instance.Set("last_error", map[string]any{
"code": code,
"message": message,
})
if err := app.Save(instance); err != nil {
app.Logger().Warn("failed to update plugin instance status", "instance", instance.Id, "error", err)
}
}
func setPluginInstanceStatusForError(app core.App, instance *core.Record, err error) {
update := pluginsystem.InstanceStatusForError(err, time.Now())
instance.Set("status", update.Status)
instance.Set("last_error", map[string]any{
"code": update.Code,
"message": update.Message,
})
if update.RetryNotBefore != nil {
instance.Set("retry_not_before", *update.RetryNotBefore)
} else {
instance.Set("retry_not_before", "")
}
if saveErr := app.Save(instance); saveErr != nil {
app.Logger().Warn("failed to update plugin instance status", "instance", instance.Id, "error", saveErr)
}
}
func applyHostPolicy(item *pluginsystem.TrailImport, config map[string]any) {
privacyMode, ok := config["privacy"].(string)
if !ok || privacyMode == "" {
privacyMode = "original"
}
if privacyMode != "original" {
item.Privacy = nil
}
}
func autoMergeEnabled(config map[string]any) bool {
merge, ok := config["merge"].(map[string]any)
return ok && boolOption(merge, "available", true) && boolOption(merge, "enabled", false)
}
func boolOption(config map[string]any, key string, fallback bool) bool {
value, ok := config[key].(bool)
if !ok {
return fallback
}
return value
}
func categoryMapping(config map[string]any) map[string]string {
raw, ok := config["categoryMapping"].(map[string]any)
if !ok {
return nil
}
result := make(map[string]string, len(raw))
for key, value := range raw {
category, ok := value.(string)
if ok {
result[key] = category
}
}
return result
}
func hasUsableCategoryMapping(mapping map[string]string) bool {
for _, category := range mapping {
if strings.TrimSpace(category) != "" {
return true
}
}
return false
}
func userDefaultPublic(app core.App, userID string) bool {
settings, err := app.FindFirstRecordByData("settings", "user", userID)
if err != nil || settings == nil {
return false
}
privacySettings := struct {
Trails string `json:"trails"`
}{}
if err := settings.UnmarshalJSONField("privacy", &privacySettings); err != nil {
return false
}
return privacySettings.Trails == "public"
}

35
db/routes/plugin_system_sync_test.go Normal file
View File

@@ -0,0 +1,35 @@
package routes
import "testing"
func TestCategoryMappingPreservesExplicitEmptyMap(t *testing.T) {
mapping := categoryMapping(map[string]any{
"categoryMapping": map[string]any{},
})
if mapping == nil {
t.Fatal("expected explicit empty category mapping to be preserved")
}
if len(mapping) != 0 {
t.Fatalf("expected empty category mapping, got %#v", mapping)
}
}
func TestCategoryMappingNilWhenMissing(t *testing.T) {
if mapping := categoryMapping(map[string]any{}); mapping != nil {
t.Fatalf("expected missing category mapping to be nil, got %#v", mapping)
}
}
func TestCategoryMappingPreservesBlankProviderMapping(t *testing.T) {
mapping := categoryMapping(map[string]any{
"categoryMapping": map[string]any{
"Ride": "",
},
})
if mapping == nil {
t.Fatal("expected category mapping")
}
if value, ok := mapping["Ride"]; !ok || value != "" {
t.Fatalf("expected blank provider mapping to be preserved, got %#v", mapping)
}
}

43
db/routes/remote_list.go
View File

@@ -49,10 +49,25 @@ func RemoteListGet(e *core.RequestEvent) error {
}
return e.InternalServerError("Sync failed", err)
}
if record.Id == "" {
// Local content that does not exist: performFullListSync
// short-circuits local IRIs and returns the unsaved shell —
// surface a real 404 instead of access/expand on a missing record.
return e.NotFoundError("List not found", nil)
}
} else {
updatedAt := record.GetDateTime("updated").Time()
if time.Now().UTC().Sub(updatedAt) > 60*time.Minute {
go performFullListSync(e.App, ctx, e.Request.URL, record)
iri := record.GetString("iri")
if time.Now().UTC().Sub(updatedAt) > remoteSyncThreshold {
if _, alreadySyncing := listSyncing.LoadOrStore(iri, struct{}{}); !alreadySyncing {
urlCopy := *e.Request.URL
bgCtx := context.WithValue(context.Background(), "actor", ctx.Value("actor"))
go func() {
defer listSyncing.Delete(iri)
performFullListSync(e.App, bgCtx, &urlCopy, record)
}()
}
}
}
} else {
@@ -102,11 +117,24 @@ func findLocalListByRemoteInfo(e *core.RequestEvent, ctx context.Context, handle
}
func performFullListSync(app core.App, ctx context.Context, reqURL *url.URL, localList *core.Record) (*core.Record, error) {
client := util.SafeHTTPClient()
iri := localList.GetString("iri")
// Never federate with ourselves (see performFullSync in remote_trail.go).
if iri == "" || util.IsLocalIRI(iri) {
if localList.GetBool("needs_full_sync") {
localList.Set("needs_full_sync", false)
if err := app.Save(localList); err != nil {
return localList, err
}
}
return localList, nil
}
client := util.SafeHTTPClient()
remoteUrl, _ := url.Parse(iri)
remoteUrl.RawQuery = reqURL.RawQuery
query := reqURL.Query()
query.Del("handle")
remoteUrl.RawQuery = query.Encode()
origin := fmt.Sprintf("%s://%s", remoteUrl.Scheme, remoteUrl.Host)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, remoteUrl.String(), nil)
@@ -115,10 +143,13 @@ func performFullListSync(app core.App, ctx context.Context, reqURL *url.URL, loc
}
res, err := client.Do(req)
if err != nil || res.StatusCode != 200 {
if err != nil {
return localList, err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return localList, fmt.Errorf("remote list fetch %s returned: %d", remoteUrl.String(), res.StatusCode)
}
var remoteMap map[string]any
if err := json.NewDecoder(res.Body).Decode(&remoteMap); err != nil {

78
db/routes/remote_trail.go
View File

@@ -8,10 +8,12 @@ import (
"io"
"net/http"
"net/url"
"path"
"os"
"pocketbase/federation"
"pocketbase/util"
"strconv"
"strings"
"sync"
"time"
"github.com/pocketbase/dbx"
@@ -19,6 +21,21 @@ import (
"github.com/pocketbase/pocketbase/tools/filesystem"
)
// remoteSyncThreshold is the minimum age of a remote record before a background sync is triggered.
// Configurable via POCKETBASE_FEDERATION_SYNC_INTERVAL (minutes). Default: 60.
var remoteSyncThreshold = func() time.Duration {
if v := os.Getenv("POCKETBASE_FEDERATION_SYNC_INTERVAL"); v != "" {
if minutes, err := strconv.Atoi(v); err == nil && minutes > 0 {
return time.Duration(minutes) * time.Minute
}
}
return 60 * time.Minute
}()
// trailSyncing and listSyncing track IRIs currently being synced to prevent concurrent duplicate syncs.
var trailSyncing sync.Map
var listSyncing sync.Map
// --- Main Handler ---
func RemoteTrailGet(e *core.RequestEvent) error {
@@ -61,11 +78,27 @@ func RemoteTrailGet(e *core.RequestEvent) error {
}
return e.InternalServerError("Sync failed", err)
}
if record.Id == "" {
// Local content that does not exist (e.g. a stale URL to a
// missing local trail): performFullSync short-circuits local
// IRIs and returns the unsaved shell — surface a real 404
// instead of running access/expand on a non-existent record.
return e.NotFoundError("Trail not found", nil)
}
} else {
// We already have it locally. Show and update background.
updatedAt := record.GetDateTime("updated").Time()
if time.Now().UTC().Sub(updatedAt) > 60*time.Minute {
go performFullSync(e.App, ctx, e.Request.URL, record)
iri := record.GetString("iri")
if time.Now().UTC().Sub(updatedAt) > remoteSyncThreshold {
if _, alreadySyncing := trailSyncing.LoadOrStore(iri, struct{}{}); !alreadySyncing {
urlCopy := *e.Request.URL
bgCtx := context.WithValue(context.Background(), "actor", ctx.Value("actor"))
go func() {
defer trailSyncing.Delete(iri)
performFullSync(e.App, bgCtx, &urlCopy, record)
}()
}
}
}
} else {
@@ -93,7 +126,10 @@ func RemoteTrailGet(e *core.RequestEvent) error {
func findLocalTrailByRemoteInfo(e *core.RequestEvent, ctx context.Context, handle, trailID string) (*core.Record, error) {
// 1. Get Actor to build the IRI
actor, err := federation.GetActorByHandle(e.App, ctx, handle, false)
if err != nil {
if err != nil && !errors.Is(err, federation.ErrProfilePrivate) {
return nil, err
}
if actor == nil {
return nil, err
}
@@ -119,19 +155,38 @@ func findLocalTrailByRemoteInfo(e *core.RequestEvent, ctx context.Context, handl
// --- Core Sync Logic ---
func performFullSync(app core.App, ctx context.Context, reqURL *url.URL, localTrail *core.Record) (*core.Record, error) {
client := util.SafeHTTPClient()
iri := localTrail.GetString("iri")
// Never federate with ourselves: a trail whose IRI is empty or points back
// to this instance is local content (we are the source of truth). Syncing it
// would fetch our own origin (or fail on an empty URL); just clear the stale
// flag so the record is no longer stuck in a permanent re-sync loop.
if iri == "" || util.IsLocalIRI(iri) {
if localTrail.GetBool("needs_full_sync") {
localTrail.Set("needs_full_sync", false)
if err := app.Save(localTrail); err != nil {
return localTrail, err
}
}
return localTrail, nil
}
client := util.SafeHTTPClient()
remoteUrl, _ := url.Parse(iri)
remoteUrl.RawQuery = reqURL.RawQuery // Forward params
query := reqURL.Query()
query.Del("handle")
remoteUrl.RawQuery = query.Encode()
origin := fmt.Sprintf("%s://%s", remoteUrl.Scheme, remoteUrl.Host)
req, _ := http.NewRequestWithContext(ctx, "GET", remoteUrl.String(), nil)
res, err := client.Do(req)
if err != nil || res.StatusCode != 200 {
if err != nil {
return localTrail, err
}
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return localTrail, fmt.Errorf("remote trail fetch %s returned: %d", remoteUrl.String(), res.StatusCode)
}
var remoteMap map[string]any
if err := json.NewDecoder(res.Body).Decode(&remoteMap); err != nil {
@@ -294,13 +349,10 @@ func syncSummitLogs(txApp core.App, ctx context.Context, trail *core.Record, ori
slID, _ := raw["id"].(string)
iri, _ := raw["iri"].(string)
if iri == "" {
iri = fmt.Sprintf("%s/api/v1/summit_logs/%s", origin, slID)
iri = fmt.Sprintf("%s/api/v1/summit-log/%s", origin, slID)
}
remoteSummitLogUrl, _ := url.Parse(iri)
possibleLocalId := path.Base(remoteSummitLogUrl.Path)
sl, _ := txApp.FindFirstRecordByFilter("summit_logs", "iri={:iri} || id={:id}", dbx.Params{"id": possibleLocalId, "iri": iri})
sl, _ := txApp.FindFirstRecordByData("summit_logs", "iri", iri)
if sl == nil {
sl = core.NewRecord(col)
}

12
db/routes/remote_trail_comment.go
View File

@@ -39,8 +39,13 @@ func RemoteTrailCommentsList(e *core.RequestEvent) error {
return err
}
trailAuthor, err := e.App.FindRecordById("activitypub_actors", trail.GetString("author"))
if err != nil {
return err
}
// Sync remote data first (Fetch + Save)
if trail.GetString("iri") != "" {
if trail.GetString("iri") != "" && !trailAuthor.GetBool("is_local") {
_ = syncRemoteComments(e, trail)
}
@@ -149,11 +154,8 @@ func syncRemoteComments(e *core.RequestEvent, trail *core.Record) error {
remoteIRI = fmt.Sprintf("%s://%s/api/v1/comment/%s", u.Scheme, u.Host, remoteID)
}
remoteCommentUrl, _ := url.Parse(remoteIRI)
possibleLocalId := path.Base(remoteCommentUrl.Path)
// Find existing record by IRI or ID to avoid duplicates
commentRecord, _ := txApp.FindFirstRecordByFilter("comments", "iri={:iri} || id={:id}", dbx.Params{"id": possibleLocalId, "iri": remoteIRI})
commentRecord, _ := txApp.FindFirstRecordByData("comments", "iri", remoteIRI)
if commentRecord == nil {
commentRecord = core.NewRecord(collection)
commentRecord.Set("iri", remoteIRI)

6
db/routes/search_token.go
View File

@@ -16,7 +16,6 @@ func SearchToken(client meilisearch.ServiceManager) func(e *core.RequestEvent) e
}
if e.Auth != nil {
userId := e.Auth.Id
userActor, err := e.App.FindFirstRecordByData("activitypub_actors", "user", e.Auth.Id)
if err != nil {
return err
@@ -24,11 +23,12 @@ func SearchToken(client meilisearch.ServiceManager) func(e *core.RequestEvent) e
searchRules = map[string]any{
"lists": map[string]string{
"filter": "public = true OR author = " + userActor.Id + " OR shares = " + userId,
"filter": "public = true OR author = " + userActor.Id + " OR shares = " + userActor.Id,
},
"trails": map[string]string{
"filter": "public = true OR author = " + userActor.Id + " OR shares = " + userId,
"filter": "public = true OR author = " + userActor.Id + " OR shares = " + userActor.Id,
},
"actors": map[string]string{},
}
}

4
db/services/trailmerge/integration_merge.go → db/services/trailmerge/plugin_merge.go
View File

@@ -13,7 +13,7 @@ func TryAutoMergeImportedTrail(
ctx context.Context,
actor *core.Record,
sourceTrailID string,
settings IntegrationAutoMergeSettings,
settings PluginAutoMergeSettings,
) error {
if actor == nil || sourceTrailID == "" || !settings.Enabled {
return nil
@@ -43,5 +43,5 @@ func TryAutoMergeImportedTrail(
return nil
}
return Merge(app, client, ctx, actor, sourceTrailID, targetTrailID, DefaultIntegrationAutoMergeMergeSettings())
return Merge(app, client, ctx, actor, sourceTrailID, targetTrailID, DefaultPluginAutoMergeMergeSettings())
}

10
db/services/trailmerge/service.go
View File

@@ -46,7 +46,7 @@ type MergeSettings struct {
Likes bool `json:"likes"`
}
type IntegrationAutoMergeSettings struct {
type PluginAutoMergeSettings struct {
Enabled bool `json:"enabled"`
}
@@ -132,13 +132,13 @@ type targetSelectionResult struct {
Stats map[string]targetSelectionStats
}
func DefaultIntegrationAutoMergeSettings() IntegrationAutoMergeSettings {
return IntegrationAutoMergeSettings{
func DefaultPluginAutoMergeSettings() PluginAutoMergeSettings {
return PluginAutoMergeSettings{
Enabled: false,
}
}
func DefaultIntegrationAutoMergeMergeSettings() MergeSettings {
func DefaultPluginAutoMergeMergeSettings() MergeSettings {
return MergeSettings{
SummitLog: true,
Photos: true,
@@ -1516,7 +1516,7 @@ func buildMergedCommentText(app core.App, comment *core.Record) string {
if authorID := comment.GetString("author"); authorID != "" {
if author, err := app.FindRecordById("activitypub_actors", authorID); err == nil {
authorHandle = "@" + author.GetString("preferred_username")
if !author.GetBool("isLocal") && author.GetString("domain") != "" {
if !author.GetBool("is_local") && author.GetString("domain") != "" {
authorHandle += "@" + author.GetString("domain")
}
}

90
db/util/activitypub.go
View File

@@ -13,7 +13,6 @@ import (
"net/http"
"net/url"
"os"
"path"
"strconv"
"strings"
"time"
@@ -87,7 +86,7 @@ func ActorFromUser(app core.App, u *core.Record) (*core.Record, error) {
record.Set("outbox", id+"/outbox")
record.Set("followers", id+"/followers")
record.Set("following", id+"/following")
record.Set("isLocal", true)
record.Set("is_local", true)
record.Set("public_key", string(pubPem))
record.Set("private_key", privEncrypted)
record.Set("user", u.Id)
@@ -111,6 +110,28 @@ func generateKeyPair() (*rsa.PrivateKey, *rsa.PublicKey, error) {
return priv, pub, nil
}
// IsLocalIRI reports whether iri belongs to this instance's own ORIGIN.
// It is used to prevent the instance from federating with itself, i.e. treating
// its own content as if it were remote.
func IsLocalIRI(iri string) bool {
if iri == "" {
return false
}
origin := os.Getenv("ORIGIN")
if origin == "" {
return false
}
o, err := url.Parse(origin)
if err != nil {
return false
}
u, err := url.Parse(iri)
if err != nil {
return false
}
return strings.EqualFold(u.Host, o.Host)
}
func TrailFromActivity(activity pub.Activity, app core.App, actor *core.Record) (*core.Record, error) {
t, err := pub.ToObject(activity.Object)
if err != nil {
@@ -118,17 +139,24 @@ func TrailFromActivity(activity pub.Activity, app core.App, actor *core.Record)
}
iri := t.ID.String()
// Own content must never be ingested as if it were remote. An inbound
// activity referencing one of our own trails (e.g. an announce echoed back)
// would otherwise flag the local trail for a full sync and later make the
// instance fetch itself. Only a local actor may resolve a local object id to
// the local record; a remote actor must not be able to reference or attach
// side effects (feeds/shares/notifications) to local content by id.
if IsLocalIRI(iri) {
if !actor.GetBool("is_local") {
return nil, fmt.Errorf("refusing remote activity referencing local trail %q", iri)
}
return app.FindFirstRecordByData("trails", "iri", iri)
}
var record *core.Record
if actor.GetBool(("isLocal")) {
trailUrl, parseErr := url.Parse(iri)
if parseErr != nil {
return nil, parseErr
}
trailId := path.Base(trailUrl.Path)
record, err = app.FindRecordById("trails", trailId)
} else {
record, err = app.FindFirstRecordByData("trails", "iri", iri)
}
if err != nil {
if err == sql.ErrNoRows {
@@ -392,7 +420,7 @@ func ObjectFromTrail(app core.App, trail *core.Record, mentions *pub.ItemCollect
}
trailObject.AttributedTo = pub.IRI(trailAuthor.GetString("iri"))
trailObject.Published = trail.GetDateTime("created").Time()
trailObject.ID = pub.IRI(fmt.Sprintf("%s/api/v1/trail/%s", origin, trail.Id))
trailObject.ID = pub.IRI(trail.GetString("iri"))
trailObject.URL = pub.IRI(activityURL)
trailObject.StartTime = trail.GetDateTime("date").Time()
@@ -409,17 +437,20 @@ func ListFromActivity(activity pub.Activity, app core.App, actor *core.Record) (
}
iri := l.ID.String()
// Own content must never be ingested as if it were remote (see TrailFromActivity).
if IsLocalIRI(iri) {
if !actor.GetBool("is_local") {
return nil, fmt.Errorf("refusing remote activity referencing local list %q", iri)
}
return app.FindFirstRecordByData("lists", "iri", iri)
}
var record *core.Record
if actor.GetBool(("isLocal")) {
listURL, parseErr := url.Parse(iri)
if parseErr != nil {
return nil, parseErr
}
listId := path.Base(listURL.Path)
record, err = app.FindRecordById("lists", listId)
} else {
record, err = app.FindFirstRecordByData("lists", "iri", iri)
}
if err != nil {
if err == sql.ErrNoRows {
collection, err := app.FindCollectionByNameOrId("lists")
@@ -515,7 +546,7 @@ func ObjectFromList(app core.App, list *core.Record) (*pub.Object, error) {
listObject.AttributedTo = pub.IRI(listAuthor.GetString("iri"))
listObject.Published = list.GetDateTime("created").Time()
listObject.ID = pub.IRI(fmt.Sprintf("%s/api/v1/list/%s", origin, list.Id))
listObject.ID = pub.IRI(list.GetString("iri"))
listObject.URL = pub.IRI(activityURL)
listObject.Attachment = attachments
return listObject, nil
@@ -536,24 +567,13 @@ func ObjectFromComment(app core.App, comment *core.Record, mentions *pub.ItemCol
if err != nil {
return nil, err
}
commentTrailAuthor, err := app.FindRecordById("activitypub_actors", commentTrail.GetString("author"))
if err != nil {
return nil, err
}
trailURL := ""
if commentTrailAuthor.GetBool("isLocal") {
trailURL = fmt.Sprintf("https://%s/api/v1/trail/%s", commentTrailAuthor.GetString("domain"), comment.GetString("trail"))
} else {
trailURL = commentTrail.GetString("iri")
}
commentObject := pub.ObjectNew(pub.NoteType)
commentObject.ID = pub.IRI(fmt.Sprintf("%s/api/v1/comment/%s", origin, comment.Id))
commentObject.ID = pub.IRI(comment.GetString("iri"))
commentObject.Content = pub.NaturalLanguageValuesNew(pub.LangRefValueNew(pub.NilLangRef, comment.GetString("text")))
commentObject.Published = comment.GetDateTime("created").Time()
commentObject.AttributedTo = pub.IRI(commentAuthor.GetString("iri"))
commentObject.InReplyTo = pub.IRI(trailURL)
commentObject.InReplyTo = pub.IRI(commentTrail.GetString("iri"))
if mentions != nil {
commentObject.Tag = *mentions

133
db/util/meilisearch.go
View File

@@ -13,11 +13,9 @@ import (
"github.com/meilisearch/meilisearch-go"
"github.com/pocketbase/pocketbase/core"
"github.com/tkrajina/gpxgo/gpx"
"github.com/twpayne/go-polyline"
)
func documentFromTrailRecord(app core.App, r *core.Record, author *core.Record, includeShares bool) (map[string]interface{}, error) {
func documentFromTrailRecord(r *core.Record, author *core.Record, includeShares bool) (map[string]interface{}, error) {
photos := r.GetStringSlice("photos")
thumbnail := ""
if len(photos) > 0 {
@@ -41,16 +39,18 @@ func documentFromTrailRecord(app core.App, r *core.Record, author *core.Record,
category = trailCategory.GetString("name")
}
polyline, err := getPolyline(app, r)
if err != nil {
polyline = ""
}
bounds := getStoredBounds(r)
domain := ""
if !author.GetBool("isLocal") {
if !author.GetBool("is_local") {
domain = author.GetString("domain")
}
diagonal := r.GetFloat("bounding_box_diagonal")
if diagonal == 0 && (bounds[0] != bounds[1] || bounds[2] != bounds[3]) {
diagonal = HaversineDistance(bounds[0], bounds[2], bounds[1], bounds[3])
}
document := map[string]any{
"id": r.Id,
"author": author.Id,
@@ -72,9 +72,14 @@ func documentFromTrailRecord(app core.App, r *core.Record, author *core.Record,
"thumbnail": thumbnail,
"gpx": r.GetString("gpx"),
"tags": tags,
"polyline": polyline,
"polyline": r.GetString("polyline"),
"domain": domain,
"iri": r.GetString("iri"),
"min_lat": bounds[0],
"max_lat": bounds[1],
"min_lon": bounds[2],
"max_lon": bounds[3],
"bounding_box_diagonal": diagonal,
"_geo": map[string]float64{
"lat": r.GetFloat("lat"),
"lng": r.GetFloat("lon"),
@@ -128,44 +133,20 @@ func difficultyToNumber(difficulty string) int32 {
return 0
}
func getPolyline(app core.App, r *core.Record) (string, error) {
gpxPath := r.GetString("gpx")
if len(gpxPath) == 0 {
return "", nil
}
avatarKey := r.BaseFilesPath() + "/" + gpxPath
fsys, err := app.NewFilesystem()
if err != nil {
return "", err
}
defer fsys.Close()
func getStoredBounds(r *core.Record) [4]float64 {
lat := r.GetFloat("lat")
lon := r.GetFloat("lon")
defaultBounds := [4]float64{lat, lat, lon, lon}
gpxFile, err := fsys.GetReader(avatarKey)
if err != nil {
return "", err
}
defer gpxFile.Close()
content := new(bytes.Buffer)
_, err = io.Copy(content, gpxFile)
if err != nil {
return "", err
}
gpxData, err := gpx.Parse(content)
if err != nil {
return "", err
minLat := r.GetFloat("min_lat")
maxLat := r.GetFloat("max_lat")
minLon := r.GetFloat("min_lon")
maxLon := r.GetFloat("max_lon")
if minLat == 0 && maxLat == 0 && minLon == 0 && maxLon == 0 && (lat != 0 || lon != 0) {
return defaultBounds
}
gpxData.SimplifyTracks(50)
coordinates := make([][]float64, 4)
for _, trk := range gpxData.Tracks {
for _, seg := range trk.Segments {
for _, pt := range seg.Points {
coordinates = append(coordinates, []float64{pt.Latitude, pt.Longitude})
}
}
}
return string(polyline.EncodeCoords(coordinates)), nil
return [4]float64{minLat, maxLat, minLon, maxLon}
}
func documentFromListRecord(r *core.Record, author *core.Record, includeShares bool) (map[string]any, error) {
@@ -176,7 +157,7 @@ func documentFromListRecord(r *core.Record, author *core.Record, includeShares b
totalDuration := 0.0
trails := len(r.GetStringSlice("trails"))
if r.GetString("iri") != "" {
if r.GetString("iri") != "" && !author.GetBool("is_local") {
doc, err := documentFromRemoteRecord(r, "lists")
if err == nil {
totalElevationGain = doc["elevation_gain"].(float64)
@@ -200,7 +181,7 @@ func documentFromListRecord(r *core.Record, author *core.Record, includeShares b
}
domain := ""
if !author.GetBool("isLocal") {
if !author.GetBool("is_local") {
domain = author.GetString("domain")
}
@@ -241,6 +222,21 @@ func documentFromListRecord(r *core.Record, author *core.Record, includeShares b
return document, nil
}
func documentFromActorRecord(r *core.Record) (map[string]any, error) {
document := map[string]any{
"id": r.Id,
"username": r.GetString("username"),
"preferred_username": r.GetString("preferred_username"),
"domain": r.GetString("domain"),
"iri": r.GetString("iri"),
"icon": r.GetString("icon"),
"is_local": r.GetBool("is_local"),
}
return document, nil
}
func documentFromRemoteRecord(r *core.Record, index string) (map[string]any, error) {
client := &http.Client{}
@@ -328,7 +324,7 @@ func IndexTrails(app core.App, trails []*core.Record, client meilisearch.Service
author := r.ExpandedOne("author")
doc, err := documentFromTrailRecord(app, r, author, true)
doc, err := documentFromTrailRecord(r, author, true)
if err != nil {
return err
}
@@ -353,24 +349,16 @@ func UpdateTrail(app core.App, r *core.Record, author *core.Record, client meili
return fmt.Errorf("meilisearch update trail: failed to expand category: %v", errs)
}
doc, err := documentFromTrailRecord(app, r, author, false)
doc, err := documentFromTrailRecord(r, author, false)
if err != nil {
return err
}
documents := []map[string]interface{}{doc}
task, err := client.Index("trails").UpdateDocuments(documents, nil)
if err != nil {
if _, err = client.Index("trails").UpdateDocuments(documents, nil); err != nil {
return err
}
interval := 500 * time.Millisecond
_, err = client.WaitForTask(task.TaskUID, interval)
if err != nil {
return fmt.Errorf("meilisearch update trail: error waiting for task completion: %v", err)
}
return nil
}
@@ -451,6 +439,37 @@ func UpdateList(app core.App, r *core.Record, author *core.Record, client meilis
return nil
}
func IndexActors(actors []*core.Record, client meilisearch.ServiceManager) error {
documents := make([]map[string]any, len(actors))
for i, r := range actors {
doc, err := documentFromActorRecord(r)
if err != nil {
return err
}
documents[i] = doc
}
if _, err := client.Index("actors").AddDocuments(documents, nil); err != nil {
return err
}
return nil
}
func UpdateActor(r *core.Record, client meilisearch.ServiceManager) error {
documents, err := documentFromActorRecord(r)
if err != nil {
return err
}
if _, err = client.Index("actors").UpdateDocuments(documents, nil); err != nil {
return err
}
return nil
}
func UpdateListShares(listId string, shares []string, client meilisearch.ServiceManager) error {
documents := []map[string]interface{}{
{

68
db/util/network_test.go Normal file
View File

@@ -0,0 +1,68 @@
package util
import (
"bytes"
"context"
"net"
"testing"
)
func TestFetchPublicURLRejectsUnsafeInputs(t *testing.T) {
tests := []string{
"ftp://example.com/file.jpg",
"http://user:pass@example.com/file.jpg",
"http://127.0.0.1/file.jpg",
"http://localhost/file.jpg",
"http://10.0.0.1/file.jpg",
"http://169.254.169.254/latest/meta-data",
"http://[::1]/file.jpg",
"http://[fc00::1]/file.jpg",
"http://example.com:8080/file.jpg",
}
for _, rawURL := range tests {
t.Run(rawURL, func(t *testing.T) {
if _, err := FetchPublicURL(context.Background(), rawURL, 1024); err == nil {
t.Fatal("expected error")
}
})
}
}
func TestReadBoundedForPlugin(t *testing.T) {
if _, err := ReadBoundedForPlugin(bytes.NewReader([]byte("1234")), 4); err != nil {
t.Fatalf("unexpected exact-limit error: %v", err)
}
if _, err := ReadBoundedForPlugin(bytes.NewReader([]byte("12345")), 4); err == nil {
t.Fatal("expected oversized response error")
}
}
func TestConnectorTLSConfigRejectsInsecureMode(t *testing.T) {
if _, err := connectorTLSConfig("insecure", nil); err == nil {
t.Fatal("expected insecure TLS mode to be rejected")
}
}
func TestConnectorIPAllowed(t *testing.T) {
tests := []struct {
ip string
allowPrivate bool
want bool
}{
{ip: "8.8.8.8", want: true},
{ip: "10.0.0.1", want: false},
{ip: "10.0.0.1", allowPrivate: true, want: true},
{ip: "fc00::1", allowPrivate: true, want: true},
{ip: "127.0.0.1", allowPrivate: true, want: false},
{ip: "169.254.1.1", allowPrivate: true, want: false},
{ip: "100.64.0.1", allowPrivate: true, want: false},
{ip: "192.0.2.1", allowPrivate: true, want: false},
}
for _, test := range tests {
t.Run(test.ip, func(t *testing.T) {
if got := connectorIPAllowed(net.ParseIP(test.ip), test.allowPrivate); got != test.want {
t.Fatalf("got %v, want %v", got, test.want)
}
})
}
}

2
db/util/notification.go
View File

@@ -61,7 +61,7 @@ func SendNotification(app core.App, notification Notification, recipient *core.R
if notification.Author == recipient.Id {
return nil
}
if !recipient.GetBool("isLocal") {
if !recipient.GetBool("is_local") {
return nil
}
permissions, err := getNotificationPermissions(app, recipient.GetString("user"), notification.Type)

142
db/util/polyline.go Normal file
View File

@@ -0,0 +1,142 @@
package util
import (
"bytes"
"fmt"
"io"
"github.com/pocketbase/pocketbase/core"
"github.com/tkrajina/gpxgo/gpx"
"github.com/twpayne/go-polyline"
)
const PolylineMaxLength = 5 * 1024 * 1024
type TrailGeometry struct {
Polyline string
MinLat float64
MaxLat float64
MinLon float64
MaxLon float64
BoundingBoxDiagonal float64
}
func ComputeTrailGeometry(app core.App, r *core.Record) (*TrailGeometry, error) {
geometry := &TrailGeometry{
MinLat: r.GetFloat("lat"),
MaxLat: r.GetFloat("lat"),
MinLon: r.GetFloat("lon"),
MaxLon: r.GetFloat("lon"),
}
gpxPath := r.GetString("gpx")
if len(gpxPath) == 0 {
return geometry, nil
}
fsys, err := app.NewFilesystem()
if err != nil {
return nil, fmt.Errorf("open filesystem: %w", err)
}
defer fsys.Close()
gpxFilePath := r.BaseFilesPath() + "/" + gpxPath
gpxFile, err := fsys.GetReader(gpxFilePath)
if err != nil {
return nil, fmt.Errorf("open gpx file %q: %w", gpxFilePath, err)
}
defer gpxFile.Close()
content := new(bytes.Buffer)
if _, err = io.Copy(content, gpxFile); err != nil {
return nil, fmt.Errorf("read gpx file %q: %w", gpxFilePath, err)
}
gpxData, err := gpx.Parse(content)
if err != nil {
return nil, fmt.Errorf("parse gpx file %q: %w", gpxFilePath, err)
}
minLat, maxLat, minLon, maxLon := 90.0, -90.0, 180.0, -180.0
hasPoints := false
addPoint := func(lat, lon float64) {
if lat < minLat {
minLat = lat
}
if lat > maxLat {
maxLat = lat
}
if lon < minLon {
minLon = lon
}
if lon > maxLon {
maxLon = lon
}
hasPoints = true
}
for _, trk := range gpxData.Tracks {
for _, seg := range trk.Segments {
for _, pt := range seg.Points {
addPoint(pt.Latitude, pt.Longitude)
}
}
}
for _, rte := range gpxData.Routes {
for _, pt := range rte.Points {
addPoint(pt.Latitude, pt.Longitude)
}
}
gpxData.SimplifyTracks(50)
coordinates := make([][]float64, 0)
for _, trk := range gpxData.Tracks {
for _, seg := range trk.Segments {
for _, pt := range seg.Points {
coordinates = append(coordinates, []float64{pt.Latitude, pt.Longitude})
}
}
}
geometry.Polyline = string(polyline.EncodeCoords(coordinates))
if hasPoints {
geometry.MinLat = minLat
geometry.MaxLat = maxLat
geometry.MinLon = minLon
geometry.MaxLon = maxLon
geometry.BoundingBoxDiagonal = HaversineDistance(minLat, minLon, maxLat, maxLon)
}
return geometry, nil
}
func ComputePolyline(app core.App, r *core.Record) (string, error) {
geometry, err := ComputeTrailGeometry(app, r)
if err != nil {
return "", err
}
return geometry.Polyline, nil
}
func SavePolyline(app core.App, r *core.Record) error {
geometry, err := ComputeTrailGeometry(app, r)
if err != nil {
return err
}
// Encoded polylines are ASCII-only, so byte length matches character length.
if len(geometry.Polyline) > PolylineMaxLength {
return fmt.Errorf("polyline exceeds maximum length of %d characters", PolylineMaxLength)
}
r.Set("polyline", geometry.Polyline)
r.Set("min_lat", geometry.MinLat)
r.Set("max_lat", geometry.MaxLat)
r.Set("min_lon", geometry.MinLon)
r.Set("max_lon", geometry.MaxLon)
r.Set("bounding_box_diagonal", geometry.BoundingBoxDiagonal)
if err := app.UnsafeWithoutHooks().Save(r); err != nil {
return fmt.Errorf("save trail geometry: %w", err)
}
return nil
}

226
db/util/safe_fetch.go Normal file
View File

@@ -0,0 +1,226 @@
package util
import (
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"io"
"net"
"net/http"
"net/netip"
"net/url"
"time"
"github.com/doyensec/safeurl"
)
const (
DefaultPluginMediaMaxBytes int64 = 50 << 20
DefaultPluginMaxImportMediaItems = 20
DefaultPluginMaxImportMediaBytes int64 = 200 << 20
)
type SafeFetchResult struct {
Body []byte
ContentType string
FinalURL string
}
type ConnectorHTTPPolicy struct {
BaseURL string
AllowPrivate bool
TLSMode string
TLSCABundle []byte
}
func FetchPublicURL(ctx context.Context, rawURL string, maxBytes int64) (*SafeFetchResult, error) {
if maxBytes <= 0 {
maxBytes = DefaultPluginMediaMaxBytes
}
parsed, err := url.Parse(rawURL)
if err != nil || parsed.Scheme == "" || parsed.Host == "" {
return nil, fmt.Errorf("invalid public URL")
}
if parsed.User != nil {
return nil, fmt.Errorf("public URL must not include credentials")
}
config := safeurl.GetConfigBuilder().
SetTimeout(60*time.Second).
SetAllowedSchemes("http", "https").
SetAllowedPorts(80, 443).
EnableIPv6(true).
AllowSendingCredentials(false).
SetCheckRedirect(publicMediaRedirectPolicy).
Build()
client := safeurl.Client(config)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
if err != nil {
return nil, err
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ReadBoundedForPlugin(resp.Body, maxBytes)
if err != nil {
return nil, err
}
return &SafeFetchResult{
Body: body,
ContentType: resp.Header.Get("Content-Type"),
FinalURL: resp.Request.URL.String(),
}, nil
}
func publicMediaRedirectPolicy(req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return fmt.Errorf("too many redirects")
}
if req.URL.User != nil {
return fmt.Errorf("redirect URL must not include credentials")
}
if req.URL.Scheme != "http" && req.URL.Scheme != "https" {
return fmt.Errorf("redirect scheme must be http or https")
}
if len(via) > 0 && via[len(via)-1].URL.Scheme == "https" && req.URL.Scheme == "http" {
return fmt.Errorf("redirect downgrades https to http")
}
return nil
}
func ConnectorHTTPClient(policy ConnectorHTTPPolicy, checkRedirect func(req *http.Request, via []*http.Request) error) (*http.Client, error) {
base, err := url.Parse(policy.BaseURL)
if err != nil || base.Scheme == "" || base.Host == "" {
return nil, fmt.Errorf("invalid connector baseURL")
}
tlsConfig, err := connectorTLSConfig(policy.TLSMode, policy.TLSCABundle)
if err != nil {
return nil, err
}
dialer := &net.Dialer{Timeout: 30 * time.Second}
transport := &http.Transport{
TLSClientConfig: tlsConfig,
DialContext: func(ctx context.Context, network string, addr string) (net.Conn, error) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
return nil, err
}
ips, err := net.DefaultResolver.LookupIP(ctx, "ip", host)
if err != nil || len(ips) == 0 {
return nil, fmt.Errorf("failed to resolve connector host: %w", err)
}
var selected net.IP
for _, ip := range ips {
if connectorIPAllowed(ip, policy.AllowPrivate) {
selected = ip
break
}
}
if selected == nil {
return nil, fmt.Errorf("connector host resolved outside allowed IP policy")
}
return dialer.DialContext(ctx, network, net.JoinHostPort(selected.String(), port))
},
}
return &http.Client{
Timeout: 60 * time.Second,
Transport: transport,
CheckRedirect: checkRedirect,
}, nil
}
func connectorTLSConfig(mode string, caBundle []byte) (*tls.Config, error) {
switch mode {
case "", "system":
return nil, nil
case "customCA":
roots, err := x509.SystemCertPool()
if err != nil || roots == nil {
roots = x509.NewCertPool()
}
if len(caBundle) == 0 || !roots.AppendCertsFromPEM(caBundle) {
return nil, fmt.Errorf("connector customCA bundle is invalid")
}
return &tls.Config{RootCAs: roots}, nil
default:
return nil, fmt.Errorf("unsupported connector TLS mode %q", mode)
}
}
func connectorIPAllowed(ip net.IP, allowPrivate bool) bool {
addr, ok := netip.AddrFromSlice(ip)
if !ok {
return false
}
if addr.Is4In6() {
addr = addr.Unmap()
}
if addr.IsLoopback() || addr.IsLinkLocalUnicast() || addr.IsLinkLocalMulticast() ||
addr.IsMulticast() || addr.IsUnspecified() {
return false
}
if isSpecialPurposeIP(addr) {
return false
}
if addr.IsPrivate() {
return allowPrivate
}
return true
}
func isSpecialPurposeIP(addr netip.Addr) bool {
for _, prefix := range specialPurposePrefixes {
if prefix.Contains(addr) {
return true
}
}
return false
}
var specialPurposePrefixes = mustPrefixes(
"0.0.0.0/8",
"100.64.0.0/10",
"127.0.0.0/8",
"169.254.0.0/16",
"192.0.0.0/24",
"192.0.2.0/24",
"198.18.0.0/15",
"198.51.100.0/24",
"203.0.113.0/24",
"224.0.0.0/4",
"240.0.0.0/4",
"::/128",
"::1/128",
"64:ff9b::/96",
"100::/64",
"2001:db8::/32",
"fe80::/10",
"ff00::/8",
)
func mustPrefixes(values ...string) []netip.Prefix {
prefixes := make([]netip.Prefix, 0, len(values))
for _, value := range values {
prefix, err := netip.ParsePrefix(value)
if err != nil {
panic(err)
}
prefixes = append(prefixes, prefix)
}
return prefixes
}
func ReadBoundedForPlugin(reader io.Reader, maxBytes int64) ([]byte, error) {
body, err := io.ReadAll(io.LimitReader(reader, maxBytes+1))
if err != nil {
return nil, err
}
if int64(len(body)) > maxBytes {
return nil, fmt.Errorf("response exceeds maximum size")
}
return body, nil
}

45
db/util/trail_access.go Normal file
View File

@@ -0,0 +1,45 @@
package util
import (
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
)
// TrailViewableByUser mirrors the trails view/read rule for custom backend
// routes that load a trail server-side and therefore bypass PocketBase's normal
// collection API permission checks.
func TrailViewableByUser(app core.App, trail *core.Record, userID string, shareToken string) bool {
if trail == nil || userID == "" {
return false
}
if trail.GetBool("public") {
return true
}
actor, err := app.FindFirstRecordByData("activitypub_actors", "user", userID)
if err != nil {
return false
}
if trail.GetString("author") == actor.Id {
return true
}
share, err := app.FindFirstRecordByFilter(
"trail_share",
"trail={:trail} && actor={:actor}",
dbx.Params{"trail": trail.Id, "actor": actor.Id},
)
if err == nil && share != nil {
return true
}
if shareToken == "" {
return false
}
linkShare, err := app.FindFirstRecordByFilter(
"trail_link_share",
"trail={:trail} && token={:token}",
dbx.Params{"trail": trail.Id, "token": shareToken},
)
return err == nil && linkShare != nil
}

133
db/util/trail_external_reference.go
View File

@@ -1,52 +1,144 @@
package util
import (
"database/sql"
"errors"
"fmt"
"strings"
"time"
"github.com/pocketbase/dbx"
"github.com/pocketbase/pocketbase/core"
)
func FindTrailByExternalReference(app core.App, provider string, externalID string) (*core.Record, error) {
if provider == "" || externalID == "" {
func FindTrailByExternalReferenceForUser(app core.App, userID string, provider string, externalID string) (*core.Record, error) {
if userID == "" || provider == "" || externalID == "" {
return nil, nil
}
refs, err := app.FindRecordsByFilter(
"trail_external_reference",
"provider={:provider} && external_id={:external_id}",
"user={:user} && provider={:provider} && external_id={:external_id}",
"+created",
1,
0,
dbx.Params{
"user": userID,
"provider": provider,
"external_id": externalID,
},
)
if err != nil || len(refs) == 0 {
if err != nil {
return nil, err
}
return nil, nil
}
trailID := refs[0].GetString("trail")
if trailID == "" {
return nil, nil
}
return app.FindRecordById("trails", trailID)
trail, err := app.FindRecordById("trails", trailID)
if err == nil {
return trail, nil
}
if !errors.Is(err, sql.ErrNoRows) {
return nil, err
}
func EnsureTrailExternalReference(app core.App, trailID string, provider string, externalID string) error {
if deleteErr := app.Delete(refs[0]); deleteErr != nil {
return nil, fmt.Errorf("delete orphaned trail external reference: %w", deleteErr)
}
app.Logger().Warn("deleted orphaned trail external reference", "provider", provider, "external_id", externalID, "trail", trailID)
return nil, nil
}
func FindExistingExternalReferenceIDsForUser(app core.App, userID string, provider string, externalIDs []string) (map[string]bool, error) {
existingIDs := map[string]bool{}
if userID == "" || provider == "" || len(externalIDs) == 0 {
return existingIDs, nil
}
params := dbx.Params{
"user": userID,
"provider": provider,
}
seen := map[string]bool{}
idFilters := make([]string, 0, len(externalIDs))
for _, externalID := range externalIDs {
if externalID == "" || seen[externalID] {
continue
}
seen[externalID] = true
paramName := fmt.Sprintf("external_id_%d", len(idFilters))
params[paramName] = externalID
idFilters = append(idFilters, "external_id={:"+paramName+"}")
}
if len(idFilters) == 0 {
return existingIDs, nil
}
filter := "user={:user} && provider={:provider} && (" + strings.Join(idFilters, " || ") + ")"
refs, err := app.FindRecordsByFilter("trail_external_reference", filter, "", len(idFilters), 0, params)
if err != nil || len(refs) == 0 {
return existingIDs, err
}
trailIDs := make([]string, 0, len(refs))
for _, ref := range refs {
if trailID := ref.GetString("trail"); trailID != "" {
trailIDs = append(trailIDs, trailID)
}
}
var trails []*core.Record
if len(trailIDs) > 0 {
trails, err = app.FindRecordsByIds("trails", trailIDs)
if err != nil {
return nil, err
}
}
trailsByID := make(map[string]bool, len(trails))
for _, trail := range trails {
trailsByID[trail.Id] = true
}
for _, ref := range refs {
trailID := ref.GetString("trail")
if trailID != "" && trailsByID[trailID] {
existingIDs[ref.GetString("external_id")] = true
continue
}
if deleteErr := app.Delete(ref); deleteErr != nil {
return nil, fmt.Errorf("delete orphaned trail external reference: %w", deleteErr)
}
app.Logger().Warn("deleted orphaned trail external reference", "provider", provider, "external_id", ref.GetString("external_id"), "trail", trailID)
}
return existingIDs, nil
}
func EnsureTrailExternalReference(app core.App, trailID string, provider string, externalID string, pluginID string, providerCategory string) error {
if trailID == "" || provider == "" || externalID == "" {
return nil
}
userID, err := externalReferenceUserID(app, trailID)
if err != nil {
return err
}
if userID == "" {
app.Logger().Warn("skipping trail external reference without local user", "provider", provider, "external_id", externalID, "trail", trailID)
return nil
}
refs, err := app.FindRecordsByFilter(
"trail_external_reference",
"provider={:provider} && external_id={:external_id}",
"user={:user} && provider={:provider} && external_id={:external_id}",
"",
1,
0,
dbx.Params{
"user": userID,
"provider": provider,
"external_id": externalID,
},
@@ -56,6 +148,19 @@ func EnsureTrailExternalReference(app core.App, trailID string, provider string,
}
if len(refs) > 0 {
if refs[0].GetString("trail") == trailID {
changed := false
if pluginID != "" && refs[0].GetString("plugin_id") == "" {
refs[0].Set("plugin_id", pluginID)
changed = true
}
if refs[0].GetDateTime("provider_category_checked_at").IsZero() {
refs[0].Set("provider_category", providerCategory)
refs[0].Set("provider_category_checked_at", time.Now())
changed = true
}
if changed {
return app.Save(refs[0])
}
return nil
}
return fmt.Errorf("trail external reference already exists for another trail")
@@ -69,13 +174,29 @@ func EnsureTrailExternalReference(app core.App, trailID string, provider string,
record := core.NewRecord(collection)
record.Load(map[string]any{
"trail": trailID,
"user": userID,
"provider": provider,
"external_id": externalID,
"plugin_id": pluginID,
"provider_category": providerCategory,
"provider_category_checked_at": time.Now(),
})
return app.Save(record)
}
func externalReferenceUserID(app core.App, trailID string) (string, error) {
trail, err := app.FindRecordById("trails", trailID)
if err != nil {
return "", err
}
actor, err := app.FindRecordById("activitypub_actors", trail.GetString("author"))
if err != nil {
return "", err
}
return actor.GetString("user"), nil
}
func ReassignTrailExternalReferences(app core.App, sourceTrailID string, targetTrailID string) error {
if sourceTrailID == "" || targetTrailID == "" || sourceTrailID == targetTrailID {
return nil

2
docker-compose.yml
View File

@@ -41,6 +41,7 @@ services:
restart: unless-stopped
volumes:
- ./data/pb_data:/pb_data
- ./data/plugins:/data/plugins
healthcheck:
test: ["CMD", "/curl", "--fail", "http://localhost:8090/health"]
interval: 15s
@@ -67,6 +68,7 @@ services:
OVERPASS_API_URL: https://overpass-api.de
VALHALLA_URL: https://valhalla1.openstreetmap.de
NOMINATIM_URL: https://nominatim.openstreetmap.org
PUBLIC_MAP_MAX_POLYLINES: 100
volumes:
- ./data/uploads:/app/uploads
# - ./data/about.md:/app/build/client/md/about.md

1
docker/docker-compose.dev.yml
View File

@@ -55,6 +55,7 @@ services:
OVERPASS_API_URL: https://overpass-api.de
VALHALLA_URL: https://valhalla1.openstreetmap.de
NOMINATIM_URL: https://nominatim.openstreetmap.org
PUBLIC_MAP_MAX_POLYLINES: 100
volumes:
- uploads:/app/uploads
# - ./data/about.md:/app/build/client/md/about.md

1
docker/docker-compose.prod.yml
View File

@@ -55,6 +55,7 @@ services:
OVERPASS_API_URL: https://overpass-api.de
VALHALLA_URL: https://valhalla1.openstreetmap.de
NOMINATIM_URL: https://nominatim.openstreetmap.org
PUBLIC_MAP_MAX_POLYLINES: 100
volumes:
- uploads:/app/uploads
# - ./data/about.md:/app/build/client/md/about.md

Some files were not shown because too many files have changed in this diff Show More