mirror of
https://codeberg.org/PostERG/xamxam.git
synced 2026-06-25 16:19:19 +02:00
filepond: implement async server-ID upload architecture with nested queue support + PeerTube integration
Replace `storeAsFile:true` with a full async FilePond round-trip pipeline using opaque server-side file IDs.
* Added 4 new PHP endpoints under `/admin/actions/filepond/`:
* `process.php` — upload/process single file and return opaque `file_id`
* `revert.php` — delete pending tmp uploads before form submit
* `load.php` — stream existing files by DB ID for FilePond preload
* `remove.php` — soft-delete `thesis_files` rows
* `process.php` improvements:
* accept arbitrary FilePond field names instead of hardcoded `file`
* support PHP-nested multi-file queue inputs (`queue_file[tfe][]`)
* explicit unwrapping of nested `$_FILES` structures
* add `audio/mp3` to audio + `peertube_audio` MIME whitelists
* immediate upload of `peertube_*` files to PeerTube, returning `peertube:{uuid}` IDs
* extensive `error_log()` instrumentation for request, CSRF, MIME, upload, and save stages
* `revert.php` now accepts `peertube:` IDs without local cleanup
* `ThesisFileHandler`:
* add `handleFilePondQueueFiles()` + `handleFilePondSingleFile()`
* process async uploads from `storage/tmp/filepond/` via opaque `file_id`
* inline handling of `peertube:{uuid}` IDs with direct `thesis_files` insertion
* remove obsolete deferred PeerTube queue-processing flow
* `ThesisCreateController` + `ThesisEditController`:
* gate async path behind `filepond_mode=1`
* preserve legacy multipart flow as fallback
* `file-upload-filepond.js`:
* remove `storeAsFile:true`
* add `buildServerConfig()` for async endpoint wiring
* fix `syncOrderInput()` to use `serverId`
* add `onprocessfile` hook
* add `fileValidateSizeFilterItem` for per-extension size caps
* preload existing uploads via `data-existing-files` + `server.load`
* replace static `INPUT_ID_TO_TYPE` map with `data-queue-type`
* add extensive `console.log()` debugging across upload pipeline stages
* `upload-progress.js`:
* block form submission while uploads are pending
* update `collectFileNames()` to read processed FilePond items
* Templates/layout:
* add `data-queue-type`
* add `data-existing-files`
* add global CSRF meta tag outside admin-only context
* add `filepond_mode` hidden input
* add CSRF token/meta support for partage pages
* move website URL field below file upload block
* `.gitignore`: exclude `storage/tmp/` from version control
This commit is contained in:
130
TODO.md
130
TODO.md
@@ -1,123 +1,9 @@
|
||||
# TODO
|
||||
# FilePond Server-ID Refactor
|
||||
|
||||
## Move Restrictions d'accès aux fichiers to acces.php
|
||||
|
||||
- [x] Remove fieldset from templates/admin/contenus.php
|
||||
- [x] Add fieldset to templates/admin/acces.php
|
||||
- [x] Load $siteSettings in admin/acces.php controller
|
||||
- [x] Update redirect in settings.php for formulaire_restrictions → /admin/acces.php
|
||||
|
||||
## Fix PeerTube upload — Google-resumable protocol adherence
|
||||
|
||||
- [x] Use Location header from init response (not reconstruct URL from JSON body)
|
||||
- [x] Switch chunk method from PUT → PATCH (Google-resumable variant)
|
||||
- [x] Use actual file MIME type in chunk Content-Type (not application/octet-stream)
|
||||
- [x] Ensure chunk size is multiple of 256 KB
|
||||
- [x] Add PATCH/HEAD methods to httpRequest()
|
||||
- [x] Add CURLOPT_HEADERFUNCTION to capture response headers
|
||||
- [x] Disable CURLOPT_FOLLOWLOCATION to preserve Location header
|
||||
- [x] Add cancelUpload() helper for Delete-on-error cleanup
|
||||
- [x] PeerTube upload fixed — simple multipart POST /api/v1/videos/upload works
|
||||
- [x] Upload progress: 0-25% browser upload, 25-99% server polling via /admin/actions/upload-progress.php
|
||||
- [x] Decorelate formats from fichiers: no HTMX toggling; Site web/Vidéo/Audio always visible
|
||||
- [x] Sticky formats fieldset inside parent container
|
||||
- [x] Server-side progress: PeerTubeService writes to temp file, client polls progress endpoint
|
||||
- [x] Fix cover deletion bug: !empty() instead of isset()
|
||||
- [x] Remove old duplicate file list CSS; unified recap+edit file figure styling
|
||||
- [x] Standardise uploaded/preexisting files styling: recap now matches edit (classes, icons, meta row, display_label)
|
||||
- [x] Refactor licence fieldset: Libre→CC2r+licence, Interne→opt-in licence, Interdit→none
|
||||
|
||||
## HTMX Toast Feedback for Settings Checkboxes (contenus.php)
|
||||
|
||||
- [x] Add `hx-target` response divs to the three fieldsets in contenus.php
|
||||
- [x] Update settings.php to return HTML toast on HTMX requests
|
||||
|
||||
## JS Refactoring — Extract inline scripts into app/ files
|
||||
|
||||
- [x] Remove overtype-webcomponent.min.js (unused)
|
||||
- [x] Extract copyLogContent + fallbackCopy + tab-updater → app/admin-logs.js
|
||||
- [x] Extract copyUrl → app/clipboard.js
|
||||
- [x] Extract tag-search inline script → app/pill-search.js (generalized for tag + language)
|
||||
- [x] Extract tfe.php access-request form → app/access-request.js
|
||||
- [x] Update all templates to use new external JS files
|
||||
|
||||
## Production Error Fixes (2026-05-11 remote logs)
|
||||
|
||||
- [x] **413 Request Entity Too Large** — bumped `client_max_body_size` to 256M, PHP post/upload to 256M, timeouts to 300s
|
||||
- [x] **Missing `v_smtp_active` view** on server — made all `CREATE VIEW` statements idempotent with `IF NOT EXISTS` in schema.sql
|
||||
- [x] **`bars.svg` 404** — created `app/public/assets/img/bars.svg` (animated SVG spinner)
|
||||
- [x] **Nginx rate limiting too aggressive** — increased admin zone to 300r/m, burst=30 to handle ~11 concurrent HTMX fragment requests on contenus.php page load
|
||||
- [x] **Migration idempotency** — `CREATE INDEX` / `CREATE TRIGGER` / `CREATE VIEW` now use `IF NOT EXISTS` in schema.sql and generate-schema.py; migrate.sh no longer fails on re-run
|
||||
- [ ] **Database readonly** — intermittent permission issue after deploy (added deploy-nginx recipe; permissions should be fixed by --chown + deploy-server.sh)
|
||||
- [x] **Upload progress bar not visible** — `collectFileNames()` now also checks FilePond instances directly (not just `input.files`); `upload-progress.php` no longer requires admin auth (blocked partage form polling)
|
||||
- [x] **Validation error messages hidden by generic fallback** — `ErrorHandler::userMessage` doesn't handle plain `Exception`, so all `throw new Exception(...)` validation messages fall through to "Une erreur inattendue…". Fix: add `Exception` pass-through before the generic fallback.
|
||||
- [x] **Validation messages use wrong field names** — "Nom/Prénom/Pseudo" → "Auteur·ice(s)", "Titre du mémoire" → "Titre du TFE" to match form labels. Updated autofocusFieldForError in both controllers.
|
||||
|
||||
## PeerTube Alternate Labels & FilePond Pools
|
||||
|
||||
- [x] Add `peertube_video_label` and `peertube_audio_label` columns (migration 029)
|
||||
- [x] Update PeerTubeService getSettings/updateSettings for new fields
|
||||
- [x] Add label fields to parametres.php admin form
|
||||
- [x] Handle label saving in admin/actions/settings.php
|
||||
- [x] Uncomment video/audio slots in fichiers-fragment.php with FilePond pools when PeerTube enabled
|
||||
- [x] Register `peertube_video` / `peertube_audio` queue types in file-upload-filepond.js
|
||||
- [x] Update handlePeerTubeUpload → handlePeerTubeQueueFiles in both create/edit controllers
|
||||
- [x] When PeerTube active, restrict TFE pool to PDF/images/VTT/archives only (no video/audio)
|
||||
- [x] Add HTMX swap attributes to Vidéo/Audio format checkboxes for live toggling
|
||||
- [x] Store PeerTube uploads as `peertube_ids:{uuid}` in thesis_files.file_path
|
||||
- [x] Create `templates/partials/peertube-embed.php` iframe embed template
|
||||
- [x] Render PeerTube embeds in public thesis view (tfe.php)
|
||||
- [x] Handle PeerTube files in admin recapitulatif.php and fichiers-fragment.php
|
||||
- [x] Shared SMTP credentials — remove username/password from peertube_settings (migration 031)
|
||||
- [x] PeerTubeService reads credentials from SmtpRelay
|
||||
- [x] OAuth client_id/secret fetched on-demand and cached in-memory (no DB storage)
|
||||
- [x] Resumable upload protocol (POST init + PUT chunks) in PeerTubeService::upload()
|
||||
- [x] Admin recapitulatif: show real PeerTube watch links (public/unlisted only)
|
||||
- [x] Optimize public thesis view: load PeerTube instance URL once before file loop
|
||||
- [ ] Test end-to-end: activate PeerTube, set labels, submit form with video/audio files
|
||||
|
||||
## SQLite Backup & Data Integrity (docs/backup-plan.md)
|
||||
|
||||
### Phase 1 — WAL Mode
|
||||
- [x] WAL mode already active (`PRAGMA journal_mode` → `wal`) — set in Database constructor
|
||||
- [ ] Verify `-wal` and `-shm` sidecar files exist after writes
|
||||
- [ ] Verify nginx/PHP write access to sidecar files on server
|
||||
- [x] Add deploy-verify-permissions recipe that checks ownership, directory perms, file perms, and writability after rsync
|
||||
- [x] deploy recipe now uploads and runs deploy-server.sh to fix permissions, then verifies them
|
||||
- [x] deploy recipe now runs migrations (scripts/migrate.sh) after ensuring DB exists
|
||||
- [x] fix migrate.sh to detect server vs local layout (no app/ subdir on server)
|
||||
- [x] regenerate schema.sql from local DB via generate-schema.py (includes v_smtp_active, all 28 migrations)
|
||||
- [x] fix generate-schema.py to include v_smtp_active (was explicitly excluded)
|
||||
|
||||
### Phase 2 — Audit Log
|
||||
- [x] `admin_audit_log` table already exists (migration 009), `AdminLogger` already writes to it
|
||||
- [x] Create the `audit_log` table for data-level audit (before/after row snapshots)
|
||||
- [x] Create `Audit.php` helper class
|
||||
- [x] Instrument all DELETE, UPDATE, INSERT operations on core tables (theses, tags, languages, thesis_files)
|
||||
- [ ] Verify by triggering a test delete and querying `SELECT * FROM audit_log ORDER BY id DESC LIMIT 5`
|
||||
|
||||
### Phase 3 — Soft Deletes
|
||||
- [x] Add `deleted_at` columns to `languages`, `tags`, `theses`
|
||||
- [x] Rebuild views `v_theses_full` and `v_theses_public` with `deleted_at IS NULL` filters
|
||||
- [x] Update `schema.sql` for fresh installs
|
||||
- [x] Replace all hard DELETEs with soft deletes (`DELETE` → `UPDATE ... SET deleted_at = ...`)
|
||||
- [x] Add `deleted_at IS NULL` to all SELECT queries touching these tables
|
||||
- [x] Add admin "Corbeille" view for soft-deleted theses with Restore and Hard Delete actions
|
||||
- [ ] Test each htmx-driven element (language search, tag search, repertoire filters) to confirm deleted entries don't appear
|
||||
- [ ] Admin: add soft-deleted tags/languages view with restore option
|
||||
|
||||
### Phase 4 — Hourly Snapshots via Cronjob
|
||||
- [x] Create `scripts/backup-sqlite.sh` (hot backup via `sqlite3 .backup`, gzip, retention pruning)
|
||||
- [x] Test locally — backup created, restores correctly
|
||||
- [x] Add `just backup-snapshot` command for local ad-hoc backups
|
||||
- [x] Deploy backup script to server (`/usr/local/bin/backup-sqlite.sh`) — `just deploy-backup-script`
|
||||
- [x] Create `/var/backups/xamxam/` directory on server — part of `just deploy-backup-cron`
|
||||
- [x] Add cron jobs (hourly 30d + daily 90d) — `just deploy-backup-cron`
|
||||
- [x] Test restore from production backup — `just test-restore <remote-gz-path>`
|
||||
- [x] Manual backup trigger — `just trigger-backup`
|
||||
- [x] Check backup log — `just deploy-check-backup-log`
|
||||
- [x] List remote backups — `just deploy-list-backups`
|
||||
- [x] One-shot deploy — `just deploy-backup` (script + cron)
|
||||
|
||||
### Phase 5 — Remote Sync *(for later)*
|
||||
- [ ] (Deferred)
|
||||
- [x] Step 1 — Build 4 PHP endpoints (process.php, revert.php, load.php, remove.php)
|
||||
- [x] Step 2 — Update ThesisFileHandler to accept file_ids instead of $_FILES
|
||||
- [x] Step 3 — Update file-upload-filepond.js (async server model + all fixes)
|
||||
- [x] Step 4 — Update templates (data-queue-type on all inputs, data-existing-files in edit)
|
||||
- [x] Step 5 — Update upload-progress.js (new collectFileNames, pending-uploads guard)
|
||||
- [ ] Step 6 — QA / integration testing
|
||||
- [ ] Step 7 — Cleanup: remove transition flags, remove INPUT_ID_TO_TYPE
|
||||
|
||||
Reference in New Issue
Block a user