mirror of
https://codeberg.org/PostERG/xamxam.git
synced 2026-05-06 19:19:19 +02:00
fix serve recipe: filter output to Development Server start + [200] requests only
refactor: unify CSS color variables across public and admin - Replace old variable structure with new standardized naming: - Background: --bg-primary, --bg-secondary, --bg-tertiary, --bg-active - Text: --text-primary, --text-secondary, --text-tertiary - Border: --border-primary, --border-secondary - Status: --success, --error, --warning - Accent: --accent-primary, --accent-secondary, --accent-foreground, --accent-muted - Remove admin-specific color variables (--admin-*) - Update all CSS files to use shared variables: - variables.css, common.css, main.css, admin.css - tfe.css, search.css, apropos.css, system.css, colors.css
This commit is contained in:
Pontoporeia
176
docs/ANALYSIS_PHP_VS_FLASK.md
Normal file
176
docs/ANALYSIS_PHP_VS_FLASK.md
Normal file
@@ -0,0 +1,176 @@
|
||||
# Posterg: PHP vs Flask Analysis
|
||||
|
||||
## Current Architecture Summary
|
||||
|
||||
- **Stack**: Vanilla PHP (no framework), SQLite, nginx + php-fpm
|
||||
- **Codebase**: ~9,100 lines across 48 PHP files
|
||||
- **Structure**: File-based routing (`public/` = webroot), shared templates via `include`, singleton `Database` class (1,294 lines), custom auth, rate limiting, media proxy
|
||||
- **Pages**: 8 public pages, 17 admin pages (11 views + 7 action handlers)
|
||||
- **Templating**: Raw PHP includes with variable scoping (`$isAdmin`, `$bodyClass`, `$extraCss`, etc.)
|
||||
- **Database**: SQLite via PDO, WAL mode, 13 tables, 2 views, 6 junction tables
|
||||
|
||||
---
|
||||
|
||||
## Templating
|
||||
|
||||
### Current PHP Pain Points
|
||||
|
||||
1. **No template inheritance.** Every page manually sets variables (`$pageTitle`, `$bodyClass`, `$extraCss`, `$ogTags`, `$isAdmin`) then `include`s `head.php`, `header.php`, and `footer.php` in sequence. The head template uses conditionals to branch between admin/public modes — functional but brittle.
|
||||
|
||||
2. **Variable scoping is implicit.** Templates read variables from the caller's scope. There's no contract — if `$availableYears` isn't set before `footer.php` is included, it silently renders nothing. Flask's Jinja2 would make this explicit via `render_template('page.html', years=years)`.
|
||||
|
||||
3. **No block/slot system.** The admin footer injects `$extraJs` / `$extraJsInline` via loose conventions. In Jinja2, `{% block scripts %}` handles this cleanly with override semantics.
|
||||
|
||||
4. **Repeated boilerplate.** Every page repeats the same 5-line preamble: require bootstrap, require Database, set template vars, include head, include header. A Flask `@app.route` + `render_template` collapses this to ~3 lines.
|
||||
|
||||
5. **HTML mixed with logic.** Files like `search.php` (220 lines) interleave DB queries, input validation, pagination math, OG tag construction, and HTML rendering in a single file. Flask naturally separates route handlers from templates.
|
||||
|
||||
### What Flask/Jinja2 Would Improve
|
||||
|
||||
- **Template inheritance**: One `base.html` with `{% block content %}`, `{% block head_extra %}`, `{% block scripts %}`. Admin extends `admin_base.html` which extends `base.html`.
|
||||
- **Macros**: The card rendering loop, pagination nav, and filter dropdowns are all repeated patterns that become `{% macro card(item) %}`.
|
||||
- **Auto-escaping**: Jinja2 escapes by default. The current code manually calls `htmlspecialchars()` ~150 times across the project. One missed call = XSS.
|
||||
- **Explicit context**: `render_template('tfe.html', thesis=data, files=files)` is self-documenting. The current `$data` / `$thesis` / `$item` naming is inconsistent across pages.
|
||||
|
||||
### What Flask Would NOT Improve
|
||||
|
||||
- The templates themselves would be roughly the same size — HTML is HTML.
|
||||
- The OG tag logic in `head.php` is already centralized; Jinja2 wouldn't simplify the conditional logic, just change its syntax.
|
||||
|
||||
---
|
||||
|
||||
## Routing & Code Organization
|
||||
|
||||
### Current State
|
||||
|
||||
File-based routing via nginx → `public/*.php`. Each file is a standalone entry point:
|
||||
```
|
||||
public/index.php → home page
|
||||
public/search.php → search/repertoire
|
||||
public/tfe.php → thesis detail
|
||||
public/admin/edit.php → edit form (GET)
|
||||
public/admin/actions/edit.php → edit handler (POST)
|
||||
```
|
||||
|
||||
This is simple and transparent — the URL *is* the file path. But it means:
|
||||
- No centralized middleware (auth, CSRF, rate limiting are manually required per-file)
|
||||
- No URL generation (hardcoded `href="/admin/edit.php?id=..."` everywhere)
|
||||
- POST handlers are separate files that redirect back, duplicating auth/CSRF boilerplate
|
||||
|
||||
### Flask Equivalent
|
||||
|
||||
```python
|
||||
@app.route('/admin/edit/<int:id>', methods=['GET', 'POST'])
|
||||
@login_required
|
||||
def admin_edit(id):
|
||||
if request.method == 'POST':
|
||||
...
|
||||
return redirect(url_for('admin_edit', id=id))
|
||||
return render_template('admin/edit.html', thesis=thesis)
|
||||
```
|
||||
|
||||
- `@login_required` replaces 7 identical `AdminAuth::requireLogin()` calls + 7 identical CSRF checks
|
||||
- `url_for()` replaces ~50 hardcoded URL strings
|
||||
- GET/POST in one function eliminates the `actions/` directory pattern
|
||||
|
||||
---
|
||||
|
||||
## Performance
|
||||
|
||||
### Where PHP Already Wins
|
||||
|
||||
1. **Process-per-request model with OPcache.** PHP-FPM with OPcache compiles PHP to bytecode once, then serves each request from shared memory. There is no framework initialization overhead because *there is no framework*. Each request loads only the files it needs.
|
||||
|
||||
2. **SQLite + WAL mode.** The database is local, on-disk, zero-network-hop. The `PRAGMA` settings (WAL, 8MB cache, synchronous=NORMAL) are well-tuned. This is identical regardless of language.
|
||||
|
||||
3. **Low memory footprint.** Each PHP-FPM worker uses ~10-20MB. A Flask process (gunicorn worker) with SQLAlchemy loaded uses ~30-50MB.
|
||||
|
||||
4. **No ORM overhead.** Raw PDO queries with manual bindings are as fast as it gets for SQLite. Flask would likely introduce SQLAlchemy, adding per-query overhead (object hydration, identity map, unit of work tracking).
|
||||
|
||||
5. **Static file serving by nginx.** CSS/JS/fonts are served directly by nginx, never touching PHP. This is identical with Flask behind nginx.
|
||||
|
||||
### Where Flask Would Be Comparable
|
||||
|
||||
| Aspect | PHP (current) | Flask |
|
||||
|--------|--------------|-------|
|
||||
| Cold start | ~5ms (OPcache hit) | ~50-100ms (Python import) |
|
||||
| Warm request | ~2-5ms | ~3-8ms |
|
||||
| SQLite query | Same PDO overhead | Same sqlite3/aiosqlite |
|
||||
| Template render | PHP native | Jinja2 compiled (comparable) |
|
||||
| Concurrency | php-fpm pool (sync) | gunicorn workers (sync) |
|
||||
|
||||
### Where Flask Would Be Worse
|
||||
|
||||
1. **Python is slower for raw computation.** Not relevant here — the bottleneck is SQLite I/O, not CPU.
|
||||
|
||||
2. **No equivalent to OPcache.** Python caches `.pyc` bytecode files, but Jinja2 templates must be compiled at startup or on first access. PHP OPcache stores compiled opcodes in shared memory — inherently faster for the "compile once, serve many" pattern.
|
||||
|
||||
3. **GIL.** Python's GIL limits true parallelism per process. PHP-FPM workers are independent processes with no shared lock. For a database-bound app this is irrelevant, but under high concurrency PHP-FPM scales more linearly.
|
||||
|
||||
4. **Memory per worker.** Flask + dependencies (Werkzeug, Jinja2, click, itsdangerous, markupsafe, plus any ORM) consumes more baseline memory than a PHP-FPM worker running vanilla PHP.
|
||||
|
||||
### Where Flask Would Be Better (Performance)
|
||||
|
||||
1. **Application-level caching.** Flask can hold objects in memory across requests (e.g., cached orientation lists, available years). PHP re-queries these on every request because it shares nothing between requests by default. However, PHP can use APCu for this — it's just not implemented here.
|
||||
|
||||
2. **Connection pooling.** Flask can maintain a persistent SQLite connection per worker. PHP opens a new PDO connection per request (the singleton is per-request, not per-process). For SQLite this overhead is minimal (~0.1ms), but it exists.
|
||||
|
||||
### Verdict: Performance
|
||||
|
||||
**PHP wins marginally for this specific workload.** The app is a low-traffic academic catalogue with SQLite. The differences are in the single-digit millisecond range and completely irrelevant at the expected scale (likely <100 requests/minute). Neither choice would ever be the bottleneck — the network round-trip to the user dwarfs everything.
|
||||
|
||||
---
|
||||
|
||||
## Developer Experience & Maintainability
|
||||
|
||||
### Where Flask Would Be Clearly Better
|
||||
|
||||
1. **Dependency management.** `pip install flask` + `requirements.txt` (or `pyproject.toml`). Currently there are zero external PHP dependencies — which sounds like a feature until you realize the project vendors `Parsedown.php` (2,000 lines of Markdown parsing) instead of using Composer.
|
||||
|
||||
2. **Form handling.** Flask-WTF provides declarative form classes with validation, CSRF built-in, and type coercion. The current code manually validates ~15 fields per form with inconsistent approaches (`filter_var`, `intval`, `trim`, `sanitize_string`).
|
||||
|
||||
3. **Testing.** Flask has a built-in test client (`app.test_client()`) that can simulate full request/response cycles. The current test suite uses a custom `run-tests.php` harness — functional but non-standard.
|
||||
|
||||
4. **Error handling.** Flask has `@app.errorhandler(404)`, `@app.errorhandler(500)`. The current code uses scattered `die()` calls and inconsistent error responses.
|
||||
|
||||
5. **Session management.** Flask-Login provides remember-me, session expiry, next-URL redirect after login. `AdminAuth.php` reimplements a subset of this in 121 lines.
|
||||
|
||||
### Where PHP Is Adequate or Better for This Project
|
||||
|
||||
1. **Zero build step.** Edit a `.php` file, refresh browser. No `flask run`, no virtual environment, no `pip install`. The `php -S localhost:8000` dev server with live-reload is already configured.
|
||||
|
||||
2. **Deployment simplicity.** rsync files to server, done. No virtualenv, no systemd unit for gunicorn, no WSGI/ASGI configuration. PHP-FPM is already running on the server.
|
||||
|
||||
3. **Hosting availability.** Any shared host runs PHP. Flask requires a VPS or PaaS with Python support. However, this project already uses a dedicated server with nginx, so this is moot.
|
||||
|
||||
4. **Team knowledge.** If the maintainers know PHP, rewriting in Python is a net negative regardless of technical merit.
|
||||
|
||||
---
|
||||
|
||||
## Migration Effort
|
||||
|
||||
Rewriting this project in Flask would require:
|
||||
|
||||
| Component | Effort |
|
||||
|-----------|--------|
|
||||
| Database layer (`Database.php` → SQLAlchemy or raw sqlite3) | 2-3 days |
|
||||
| 8 public routes + templates | 2 days |
|
||||
| 17 admin routes + templates | 3-4 days |
|
||||
| Auth system (Flask-Login) | 0.5 days |
|
||||
| File upload/media serving | 1 day |
|
||||
| Rate limiting (Flask-Limiter) | 0.5 days |
|
||||
| nginx config adaptation | 0.5 days |
|
||||
| Testing | 1-2 days |
|
||||
| **Total** | **~10-14 days** |
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
**Flask would have been a better starting point** for this project — primarily for templating (Jinja2 inheritance eliminates the fragile variable-scoping pattern), routing (decorators + middleware replace per-file boilerplate), and developer ergonomics (form validation, auto-escaping, test client).
|
||||
|
||||
**Flask would not deliver better performance.** The current vanilla PHP stack is actually *slightly faster* for this workload due to OPcache efficiency and the absence of framework overhead. The difference is immaterial at this scale.
|
||||
|
||||
**A rewrite is not justified.** The project works, the architecture is coherent (if verbose), and the codebase is small enough (~9K lines) to maintain. The practical improvements Flask would bring (cleaner templates, less boilerplate, better form handling) don't outweigh the cost of a full rewrite plus the operational change from PHP-FPM to gunicorn.
|
||||
|
||||
**If starting fresh today:** Flask (or even Litestar/FastAPI with Jinja2) would be the stronger choice for a SQLite-backed catalogue app of this size. The Jinja2 templating alone would save ~20% of the current codebase.
|
||||
615
docs/REFACTORING_RECOMMENDATIONS.md
Normal file
615
docs/REFACTORING_RECOMMENDATIONS.md
Normal file
@@ -0,0 +1,615 @@
|
||||
# Posterg: Refactoring Recommendations
|
||||
|
||||
Concrete improvements to the separation between templating, routing, and backend logic — staying in PHP, no framework required.
|
||||
|
||||
---
|
||||
|
||||
## 1. Extract a Micro-Router + Middleware Pipeline
|
||||
|
||||
### Problem
|
||||
|
||||
Every file repeats the same preamble. The 7 action handlers and 17 page controllers all independently:
|
||||
|
||||
```php
|
||||
require_once __DIR__ . '/../../config/bootstrap.php';
|
||||
require_once __DIR__ . '/../../src/AdminAuth.php';
|
||||
AdminAuth::requireLogin();
|
||||
if (empty($_SESSION['csrf_token'])) {
|
||||
$_SESSION['csrf_token'] = bin2hex(random_bytes(32));
|
||||
}
|
||||
require_once __DIR__ . '/../../src/Database.php';
|
||||
```
|
||||
|
||||
This is ~6-8 identical lines per file × 24 files = ~170 lines of pure duplication. When the CSRF check pattern changes, every action handler must be updated in lockstep.
|
||||
|
||||
### Solution
|
||||
|
||||
Create `src/App.php` — a thin request dispatcher with middleware hooks:
|
||||
|
||||
```php
|
||||
// src/App.php
|
||||
class App {
|
||||
private static bool $booted = false;
|
||||
|
||||
/** Boot once per request: load Database, ensure CSRF token exists. */
|
||||
public static function boot(): Database {
|
||||
if (!self::$booted) {
|
||||
require_once APP_ROOT . '/src/Database.php';
|
||||
self::$booted = true;
|
||||
}
|
||||
if (empty($_SESSION['csrf_token'])) {
|
||||
$_SESSION['csrf_token'] = bin2hex(random_bytes(32));
|
||||
}
|
||||
return Database::getInstance();
|
||||
}
|
||||
|
||||
/** Gate for admin pages: auth + CSRF token. */
|
||||
public static function adminGuard(): Database {
|
||||
require_once APP_ROOT . '/src/AdminAuth.php';
|
||||
AdminAuth::requireLogin();
|
||||
return self::boot();
|
||||
}
|
||||
|
||||
/** Validate CSRF on POST. Call at the top of every action handler. */
|
||||
public static function verifyCsrf(): void {
|
||||
if ($_SERVER['REQUEST_METHOD'] !== 'POST'
|
||||
|| !isset($_POST['csrf_token'], $_SESSION['csrf_token'])
|
||||
|| !hash_equals($_SESSION['csrf_token'], $_POST['csrf_token'])) {
|
||||
http_response_code(403);
|
||||
exit('CSRF token invalide.');
|
||||
}
|
||||
}
|
||||
|
||||
/** Regenerate CSRF after a successful mutation. */
|
||||
public static function rotateCsrf(): void {
|
||||
$_SESSION['csrf_token'] = bin2hex(random_bytes(32));
|
||||
}
|
||||
|
||||
/** Flash a message into the session and redirect. */
|
||||
public static function redirect(string $url, ?string $success = null, ?string $error = null): never {
|
||||
if ($success) $_SESSION['success'] = $success;
|
||||
if ($error) $_SESSION['error'] = $error;
|
||||
header('Location: ' . $url);
|
||||
exit;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Every admin page becomes:
|
||||
|
||||
```php
|
||||
<?php
|
||||
require_once __DIR__ . '/../../config/bootstrap.php';
|
||||
$db = App::adminGuard();
|
||||
// ... page-specific logic
|
||||
```
|
||||
|
||||
Every action handler becomes:
|
||||
|
||||
```php
|
||||
<?php
|
||||
require_once __DIR__ . '/../../../config/bootstrap.php';
|
||||
$db = App::adminGuard();
|
||||
App::verifyCsrf();
|
||||
// ... mutation logic
|
||||
App::rotateCsrf();
|
||||
App::redirect('/admin/', success: 'Done.');
|
||||
```
|
||||
|
||||
**Impact**: Eliminates ~170 lines of duplication. Centralises the CSRF lifecycle (generate, verify, rotate) in one place. Any change to the auth or CSRF pattern is a single-file edit.
|
||||
|
||||
---
|
||||
|
||||
## 2. Separate Controllers from Templates
|
||||
|
||||
### Problem
|
||||
|
||||
Every page file (e.g. `public/index.php`, `public/search.php`, `public/admin/edit.php`) is a single file that mixes three concerns:
|
||||
|
||||
1. **Data fetching** (DB queries, input validation, pagination math)
|
||||
2. **View variable preparation** (`$pageTitle`, `$ogTags`, `$extraCss`, `$bodyClass`)
|
||||
3. **HTML rendering** (the entire template, inline)
|
||||
|
||||
`system.php` is the extreme case: 400+ lines of PHP logic (systemd checks, curl pings, disk stats, log parsing, nginx syntax highlighting) followed by 200+ lines of inline `<style>`, then 200+ lines of HTML, then 30+ lines of inline `<script>`.
|
||||
|
||||
`search.php` is another: the répertoire index view and the search results view are two entirely different pages sharing a single file because they share a URL.
|
||||
|
||||
### Solution
|
||||
|
||||
Introduce a `controllers/` directory. Each controller is a function that does the data work and returns an associative array. The page file becomes a thin bridge.
|
||||
|
||||
**Directory structure:**
|
||||
|
||||
```
|
||||
src/
|
||||
App.php ← new: middleware/helpers
|
||||
controllers/
|
||||
HomeController.php
|
||||
SearchController.php
|
||||
TfeController.php
|
||||
admin/
|
||||
ThesisListController.php
|
||||
ThesisEditController.php
|
||||
ThesisAddController.php
|
||||
TagController.php
|
||||
PageController.php
|
||||
SystemController.php
|
||||
AccountController.php
|
||||
...existing files...
|
||||
|
||||
templates/
|
||||
public/
|
||||
home.php
|
||||
search-results.php
|
||||
search-index.php
|
||||
tfe.php
|
||||
apropos.php
|
||||
licence.php
|
||||
admin/
|
||||
thesis-list.php
|
||||
thesis-edit.php
|
||||
thesis-add.php
|
||||
tags.php
|
||||
pages-list.php
|
||||
pages-edit.php
|
||||
system.php
|
||||
account.php
|
||||
...existing shared templates (head.php, header.php, footer.php)...
|
||||
```
|
||||
|
||||
**Example — `search.php` split:**
|
||||
|
||||
```php
|
||||
// src/controllers/SearchController.php
|
||||
class SearchController {
|
||||
public static function index(Database $db): array {
|
||||
// Collect params, run queries, compute pagination
|
||||
// ...all current logic from the top of search.php...
|
||||
return [
|
||||
'hasSearch' => $hasSearch,
|
||||
'results' => $results,
|
||||
'totalItems' => $totalItems,
|
||||
'totalPages' => $totalPages,
|
||||
'years' => $years,
|
||||
'orientations'=> $orientations,
|
||||
'apPrograms' => $apPrograms,
|
||||
'keywords' => $keywords,
|
||||
'students' => $students,
|
||||
'authorMap' => $authorMap,
|
||||
'page' => $page,
|
||||
'error' => $validationError,
|
||||
// Template config
|
||||
'pageTitle' => 'Répertoire – Posterg',
|
||||
'bodyClass' => 'search-body',
|
||||
'extraCss' => ['/assets/css/search.css'],
|
||||
'currentNav' => 'repertoire',
|
||||
];
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
```php
|
||||
// public/search.php (entire file)
|
||||
<?php
|
||||
require_once __DIR__ . '/../config/bootstrap.php';
|
||||
require_once APP_ROOT . '/src/RateLimit.php';
|
||||
|
||||
// Rate limiting (stays here — it's a routing concern)
|
||||
$rateLimit = new RateLimit(30, 60);
|
||||
if (!$rateLimit->check()) { /* ...429 response... */ }
|
||||
$rateLimit->sendHeaders();
|
||||
|
||||
$db = App::boot();
|
||||
require_once APP_ROOT . '/src/controllers/SearchController.php';
|
||||
$data = SearchController::index($db);
|
||||
extract($data); // populates $hasSearch, $results, $pageTitle, etc.
|
||||
|
||||
include APP_ROOT . '/templates/head.php';
|
||||
include APP_ROOT . '/templates/header.php';
|
||||
if ($hasSearch) {
|
||||
include APP_ROOT . '/templates/public/search-results.php';
|
||||
} else {
|
||||
include APP_ROOT . '/templates/public/search-index.php';
|
||||
}
|
||||
include APP_ROOT . '/templates/footer.php';
|
||||
```
|
||||
|
||||
```
|
||||
// templates/public/search-results.php
|
||||
// Pure HTML + minimal <?= ?> for output. No DB queries. No input processing.
|
||||
```
|
||||
|
||||
**Impact**: Templates become auditable for XSS — they only do output. Controllers are testable — they return arrays, no output buffering needed. The `extract()` bridge keeps the familiar variable-name convention without changing every template.
|
||||
|
||||
---
|
||||
|
||||
## 3. Introduce a `render()` Helper to Replace the 5-Line Include Chain
|
||||
|
||||
### Problem
|
||||
|
||||
Every page ends with the same sequence:
|
||||
|
||||
```php
|
||||
include APP_ROOT . '/templates/head.php';
|
||||
include APP_ROOT . '/templates/header.php';
|
||||
// ... main content ...
|
||||
include APP_ROOT . '/templates/footer.php'; // or admin/footer.php
|
||||
```
|
||||
|
||||
The admin variant also requires `$isAdmin = true; $bodyClass = 'admin-body';` to be set before the head include. Forgetting any variable or include breaks the page silently.
|
||||
|
||||
### Solution
|
||||
|
||||
Add a `render()` function to `App`:
|
||||
|
||||
```php
|
||||
// In src/App.php
|
||||
public static function render(string $template, array $vars = []): void {
|
||||
extract($vars);
|
||||
include APP_ROOT . '/templates/head.php';
|
||||
include APP_ROOT . '/templates/header.php';
|
||||
include APP_ROOT . '/templates/' . $template;
|
||||
// Choose footer based on admin flag
|
||||
if (!empty($isAdmin)) {
|
||||
include APP_ROOT . '/templates/admin/footer.php';
|
||||
} else {
|
||||
include APP_ROOT . '/templates/footer.php';
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Every page becomes:
|
||||
|
||||
```php
|
||||
// public/licence.php
|
||||
<?php
|
||||
require_once __DIR__ . '/../config/bootstrap.php';
|
||||
$db = App::boot();
|
||||
require_once APP_ROOT . '/src/controllers/LicenceController.php';
|
||||
App::render('public/licence.php', LicenceController::index($db));
|
||||
```
|
||||
|
||||
Every admin page becomes:
|
||||
|
||||
```php
|
||||
// public/admin/tags.php
|
||||
<?php
|
||||
require_once __DIR__ . '/../../config/bootstrap.php';
|
||||
$db = App::adminGuard();
|
||||
require_once APP_ROOT . '/src/controllers/admin/TagController.php';
|
||||
App::render('admin/tags.php', TagController::index($db));
|
||||
```
|
||||
|
||||
**Impact**: No more forgotten includes. The head→header→content→footer pipeline is enforced. Admin footer selection is automatic. Template variables are explicit (passed as array keys, not ambient scope).
|
||||
|
||||
---
|
||||
|
||||
## 4. Consolidate Action Handlers into Controller Methods
|
||||
|
||||
### Problem
|
||||
|
||||
The `public/admin/actions/` directory contains 7 POST-only files that each:
|
||||
|
||||
1. Require bootstrap + auth
|
||||
2. Verify CSRF
|
||||
3. Extract + validate `$_POST` data
|
||||
4. Call `Database` methods
|
||||
5. Rotate CSRF
|
||||
6. Flash a message
|
||||
7. Redirect
|
||||
|
||||
Steps 1-2 and 5-7 are identical in every file. The actual business logic (step 4) is usually 5-15 lines.
|
||||
|
||||
`actions/publish.php` (95 lines) does exactly one thing: flip `is_published` on 1-N theses. The other 80 lines are auth, CSRF, validation boilerplate.
|
||||
|
||||
### Solution
|
||||
|
||||
Merge each action into its controller as a `handlePost()` or action-specific static method:
|
||||
|
||||
```php
|
||||
// src/controllers/admin/ThesisListController.php
|
||||
class ThesisListController {
|
||||
public static function index(Database $db): array {
|
||||
// ... current admin/index.php data logic ...
|
||||
}
|
||||
|
||||
public static function publish(Database $db): never {
|
||||
App::verifyCsrf();
|
||||
$action = $_POST['action'] ?? '';
|
||||
$isBulk = !empty($_POST['bulk']);
|
||||
// ... 15 lines of actual logic ...
|
||||
App::rotateCsrf();
|
||||
App::redirect('/admin/', success: "$count TFE(s) publié(s).");
|
||||
}
|
||||
|
||||
public static function toggleMaintenance(Database $db): never {
|
||||
App::verifyCsrf();
|
||||
// ... 6 lines ...
|
||||
App::rotateCsrf();
|
||||
App::redirect('/admin/', success: 'Maintenance toggled.');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The page file handles dispatch:
|
||||
|
||||
```php
|
||||
// public/admin/index.php
|
||||
<?php
|
||||
require_once __DIR__ . '/../../config/bootstrap.php';
|
||||
$db = App::adminGuard();
|
||||
require_once APP_ROOT . '/src/controllers/admin/ThesisListController.php';
|
||||
|
||||
if ($_SERVER['REQUEST_METHOD'] === 'POST') {
|
||||
$action = $_POST['_action'] ?? '';
|
||||
match($action) {
|
||||
'publish' => ThesisListController::publish($db),
|
||||
'maintenance' => ThesisListController::toggleMaintenance($db),
|
||||
default => App::redirect('/admin/', error: 'Action inconnue.'),
|
||||
};
|
||||
}
|
||||
|
||||
App::render('admin/thesis-list.php', ThesisListController::index($db));
|
||||
```
|
||||
|
||||
**Or** keep the existing `actions/*.php` files but reduce each to 3 lines:
|
||||
|
||||
```php
|
||||
// public/admin/actions/publish.php (entire file)
|
||||
<?php
|
||||
require_once __DIR__ . '/../../../config/bootstrap.php';
|
||||
$db = App::adminGuard();
|
||||
App::verifyCsrf();
|
||||
require_once APP_ROOT . '/src/controllers/admin/ThesisListController.php';
|
||||
ThesisListController::publish($db);
|
||||
```
|
||||
|
||||
**Impact**: The `actions/` directory goes from 7 files × 50-100 lines each ≈ 500 lines, to 7 files × 5 lines each ≈ 35 lines. The business logic moves into testable controller methods.
|
||||
|
||||
---
|
||||
|
||||
## 5. Extract Inline CSS and JS from `system.php`
|
||||
|
||||
### Problem
|
||||
|
||||
`system.php` contains:
|
||||
- 180 lines of `<style>` embedded in the page
|
||||
- 40 lines of `<script>` embedded in the page
|
||||
- 12 PHP helper functions defined inline (`safeExec`, `systemdStatus`, `localHttpCheck`, `humanBytes`, `statusLabel`, `statusClass`, `readLogTail`, `logLineClass`, `nginxLineClass`)
|
||||
|
||||
This makes it the largest file in the project (500+ lines) and impossible to cache the CSS/JS independently.
|
||||
|
||||
### Solution
|
||||
|
||||
1. Move the `<style>` block → `public/assets/css/system.css`, reference it via `$extraCss`
|
||||
2. Move the `<script>` block → `public/assets/js/system.js`, reference it via `$extraJs`
|
||||
3. Move the 12 helper functions → `src/controllers/admin/SystemController.php`
|
||||
4. Move the data-gathering logic (checks array, PHP info, disk stats, log reading) into `SystemController::index()`
|
||||
|
||||
The template becomes pure HTML with `<?= ?>` interpolation.
|
||||
|
||||
**Impact**: `system.php` goes from ~500 lines to ~5 lines (boot + controller + render). The CSS/JS becomes cacheable by the browser (nginx `expires 30d` rule already exists for `.css`/`.js`). The helpers become unit-testable.
|
||||
|
||||
---
|
||||
|
||||
## 6. Introduce Template Partials for Repeated UI Patterns
|
||||
|
||||
### Problem
|
||||
|
||||
Several HTML patterns are copy-pasted across templates:
|
||||
|
||||
**Flash messages** — Identical block in `admin/index.php`, `admin/edit.php`, `admin/tags.php`, `admin/pages.php`, `admin/account.php`:
|
||||
```php
|
||||
<?php if (isset($_SESSION['error'])): ?>
|
||||
<div class="admin-alert admin-alert--error">⚠ <?= htmlspecialchars($_SESSION['error']); unset($_SESSION['error']); ?></div>
|
||||
<?php endif; ?>
|
||||
<?php if (isset($_SESSION['success'])): ?>
|
||||
<div class="admin-alert admin-alert--success">✓ <?= htmlspecialchars($_SESSION['success']); unset($_SESSION['success']); ?></div>
|
||||
<?php endif; ?>
|
||||
```
|
||||
|
||||
**Pagination** — Near-identical block in `index.php` and `search.php` (30 lines each, slightly different URL building).
|
||||
|
||||
**Select dropdowns with "selected" logic** — Repeated in `admin/add.php` and `admin/edit.php` for orientation, AP, finality, license, access type.
|
||||
|
||||
**Jury fieldset + JS** — Duplicated between `admin/add.php` and `admin/edit.php` (50+ lines of identical HTML + 20 lines of identical JS).
|
||||
|
||||
### Solution
|
||||
|
||||
Create small partial templates:
|
||||
|
||||
```
|
||||
templates/
|
||||
partials/
|
||||
flash-messages.php ← reads $_SESSION['error'] / $_SESSION['success']
|
||||
pagination.php ← receives $page, $totalPages, $baseUrl
|
||||
admin/
|
||||
select-field.php ← receives $name, $label, $options, $selected
|
||||
checkbox-list.php ← receives $name, $label, $options, $checked
|
||||
jury-fieldset.php ← receives $jury (array), outputs fieldset + JS
|
||||
```
|
||||
|
||||
**Example — `flash-messages.php`:**
|
||||
|
||||
```php
|
||||
<?php
|
||||
// templates/partials/flash-messages.php
|
||||
$_flashError = $_SESSION['error'] ?? $_SESSION['admin_error'] ?? $_SESSION['edit_error'] ?? null;
|
||||
$_flashSuccess = $_SESSION['success'] ?? $_SESSION['admin_success'] ?? $_SESSION['edit_success'] ?? null;
|
||||
unset($_SESSION['error'], $_SESSION['success'], $_SESSION['admin_error'],
|
||||
$_SESSION['admin_success'], $_SESSION['edit_error'], $_SESSION['edit_success']);
|
||||
?>
|
||||
<?php if ($_flashError): ?>
|
||||
<div class="admin-alert admin-alert--error">⚠ <?= htmlspecialchars($_flashError) ?></div>
|
||||
<?php endif; ?>
|
||||
<?php if ($_flashSuccess): ?>
|
||||
<div class="admin-alert admin-alert--success">✓ <?= htmlspecialchars($_flashSuccess) ?></div>
|
||||
<?php endif; ?>
|
||||
```
|
||||
|
||||
This also fixes a latent bug: the project uses 6 different session keys for flash messages (`error`, `success`, `admin_error`, `admin_success`, `edit_error`, `edit_success`, `form_error`). Centralising flash handling would unify these into two keys.
|
||||
|
||||
**Impact**: Eliminates ~200 lines of duplicated HTML. The jury fieldset (duplicated between add and edit) becomes a single 50-line partial. Form field partials make admin pages shorter and more consistent.
|
||||
|
||||
---
|
||||
|
||||
## 7. Unify Flash Message Keys
|
||||
|
||||
### Problem
|
||||
|
||||
The project uses 7 different session keys for flash messages across different pages:
|
||||
|
||||
| Key | Used by |
|
||||
|-----|---------|
|
||||
| `$_SESSION['error']` | `admin/index.php`, `visibility.php`, `account.php` |
|
||||
| `$_SESSION['success']` | `admin/index.php`, `visibility.php`, `pages.php`, `account.php` |
|
||||
| `$_SESSION['admin_error']` | `tags.php` |
|
||||
| `$_SESSION['admin_success']` | `tags.php` |
|
||||
| `$_SESSION['edit_error']` | `admin/edit.php` |
|
||||
| `$_SESSION['edit_success']` | `admin/edit.php` |
|
||||
| `$_SESSION['form_error']` | `admin/add.php` |
|
||||
| `$_SESSION['form_data']` | `admin/add.php` (re-population) |
|
||||
|
||||
This means flash messages can silently persist across pages if a redirect sends the user somewhere that doesn't read the matching key.
|
||||
|
||||
### Solution
|
||||
|
||||
Standardise on two keys: `$_SESSION['_flash_error']` and `$_SESSION['_flash_success']`. Consume them in the shared `flash-messages.php` partial. Add `App::flash(string $type, string $message)` helper.
|
||||
|
||||
---
|
||||
|
||||
## 8. Move OG Tag Construction into Controller Logic
|
||||
|
||||
### Problem
|
||||
|
||||
Every public page constructs `$ogTags` inline before the template, with ~15 lines of boilerplate. `tfe.php` has 25 lines of OG logic including image resolution (banner → cover → none) and description truncation.
|
||||
|
||||
### Solution
|
||||
|
||||
Move OG tag logic into each controller's return array. Create a helper:
|
||||
|
||||
```php
|
||||
// In src/App.php or a dedicated helpers file
|
||||
public static function ogTags(array $overrides = []): array {
|
||||
return array_merge([
|
||||
'type' => 'website',
|
||||
'site_name' => 'Posterg – ERG',
|
||||
'title' => 'Posterg',
|
||||
'description' => '',
|
||||
'url' => '',
|
||||
'image' => '',
|
||||
], $overrides);
|
||||
}
|
||||
```
|
||||
|
||||
The TFE controller builds the OG image resolution as part of its data preparation. The template never touches it.
|
||||
|
||||
---
|
||||
|
||||
## Summary: Proposed File Layout
|
||||
|
||||
```
|
||||
src/
|
||||
App.php ← NEW: boot, adminGuard, verifyCsrf, render, redirect, flash
|
||||
Database.php ← unchanged
|
||||
AdminAuth.php ← unchanged
|
||||
RateLimit.php ← unchanged
|
||||
Parsedown.php ← unchanged
|
||||
config.php ← unchanged
|
||||
controllers/
|
||||
HomeController.php ← extracted from public/index.php
|
||||
SearchController.php ← extracted from public/search.php
|
||||
TfeController.php ← extracted from public/tfe.php
|
||||
AproposController.php ← extracted from public/apropos.php
|
||||
LicenceController.php ← extracted from public/licence.php
|
||||
admin/
|
||||
ThesisListController.php ← extracted from admin/index.php + actions/publish.php + actions/maintenance.php
|
||||
ThesisEditController.php ← extracted from admin/edit.php + actions/edit.php
|
||||
ThesisAddController.php ← extracted from admin/add.php + actions/formulaire.php
|
||||
TagController.php ← extracted from admin/tags.php + actions/tag.php
|
||||
PageController.php ← extracted from admin/pages.php + pages-edit.php + actions/page.php
|
||||
SystemController.php ← extracted from admin/system.php (400 lines of logic)
|
||||
AccountController.php ← extracted from admin/account.php + actions/account.php
|
||||
ImportController.php ← extracted from admin/import.php
|
||||
|
||||
templates/
|
||||
head.php ← unchanged
|
||||
header.php ← unchanged
|
||||
footer.php ← unchanged
|
||||
search-bar.php ← unchanged
|
||||
admin/
|
||||
footer.php ← unchanged
|
||||
partials/
|
||||
flash-messages.php ← NEW
|
||||
pagination.php ← NEW
|
||||
admin/
|
||||
jury-fieldset.php ← NEW (deduplicated from add.php + edit.php)
|
||||
select-field.php ← NEW
|
||||
checkbox-list.php ← NEW
|
||||
public/
|
||||
home.php ← extracted HTML from index.php
|
||||
search-results.php ← extracted HTML from search.php (results view)
|
||||
search-index.php ← extracted HTML from search.php (répertoire view)
|
||||
tfe.php ← extracted HTML from tfe.php
|
||||
apropos.php ← extracted HTML from apropos.php
|
||||
licence.php ← extracted HTML from licence.php
|
||||
admin/
|
||||
thesis-list.php ← extracted HTML from admin/index.php
|
||||
thesis-edit.php ← extracted HTML from admin/edit.php
|
||||
thesis-add.php ← extracted HTML from admin/add.php
|
||||
tags.php ← extracted HTML from admin/tags.php
|
||||
pages-list.php ← extracted HTML from admin/pages.php
|
||||
pages-edit.php ← extracted HTML from admin/pages-edit.php
|
||||
system.php ← extracted HTML from admin/system.php
|
||||
account.php ← extracted HTML from admin/account.php
|
||||
|
||||
public/
|
||||
index.php ← 5-8 lines: boot, controller, render
|
||||
search.php ← 8-10 lines: boot, rate limit, controller, render
|
||||
tfe.php ← 5-8 lines
|
||||
apropos.php ← 5-8 lines
|
||||
licence.php ← 5-8 lines
|
||||
media.php ← unchanged (already clean)
|
||||
maintenance.php ← unchanged
|
||||
live-reload.php ← unchanged
|
||||
admin/
|
||||
index.php ← 8-10 lines: boot, dispatch POST or render
|
||||
edit.php ← 8-10 lines
|
||||
add.php ← 8-10 lines
|
||||
tags.php ← 6-8 lines
|
||||
pages.php ← 5-8 lines
|
||||
pages-edit.php ← 5-8 lines
|
||||
system.php ← 5-8 lines
|
||||
account.php ← 8-10 lines
|
||||
import.php ← 5-8 lines
|
||||
login.php ← unchanged
|
||||
logout.php ← unchanged
|
||||
actions/ ← can be REMOVED (merged into controllers)
|
||||
OR reduced to 3-5 line stubs
|
||||
```
|
||||
|
||||
## Estimated Impact
|
||||
|
||||
| Metric | Before | After |
|
||||
|--------|--------|-------|
|
||||
| Lines in `public/` page files | ~4,200 | ~200 |
|
||||
| Lines in `actions/` handlers | ~500 | 0 (merged) |
|
||||
| Duplicated boilerplate | ~300 lines | ~0 |
|
||||
| Duplicated HTML (jury, flash, pagination) | ~200 lines | ~0 |
|
||||
| `system.php` | 530 lines | ~8 lines (page) + ~180 (controller) + ~100 (CSS) + ~30 (JS) |
|
||||
| Testable controller methods | 0 | ~15 |
|
||||
| Flash session key variants | 7 | 2 |
|
||||
|
||||
Total net reduction: ~800–1,000 lines eliminated through deduplication, with cleaner separation making the remaining code auditable and testable.
|
||||
|
||||
## Execution Order
|
||||
|
||||
This can be done incrementally, one page at a time, with zero disruption:
|
||||
|
||||
1. **Create `src/App.php`** with `boot()`, `adminGuard()`, `verifyCsrf()`, `rotateCsrf()`, `redirect()`, `render()`, `flash()`
|
||||
2. **Create `templates/partials/flash-messages.php`** and adopt it in one admin page
|
||||
3. **Extract `SystemController`** — the biggest single-file win (500 → 8 lines)
|
||||
4. **Extract `SearchController`** — the most complex public page
|
||||
5. **Extract `ThesisEditController`** — merges `edit.php` + `actions/edit.php`, deduplicates jury fieldset
|
||||
6. **Do remaining controllers one by one**, smallest first
|
||||
7. **Unify flash keys** project-wide as the last step
|
||||
167
docs/chat-export-2026-04-02.md
Normal file
167
docs/chat-export-2026-04-02.md
Normal file
@@ -0,0 +1,167 @@
|
||||
### User
|
||||
|
||||
for a low traffic website, using a csv vs a sqlite or a nosql db, which would make sense ? it would be a php website.
|
||||
|
||||
### Assistant
|
||||
|
||||
Short answer:**use SQLite**for almost all low-traffic PHP sites unless your data is literally a tiny, read-only list best kept as a CSV. CSVs are fragile; NoSQL is usually overkill and adds operational cost.
|
||||
|
||||
Here’s a practical breakdown to help you choose.
|
||||
|
||||
# When to pick each
|
||||
|
||||
**SQLite (recommended)**
|
||||
|
||||
- Best for low-traffic sites with structured data (users, posts, settings, relations).
|
||||
- ACID transactions, SQL queries, indexes, easy backups (single file).
|
||||
- Very simple to use from PHP via PDO; no separate server to manage.
|
||||
- Handles many concurrent reads well; writes are serialized but fine for low write volume. Enable WAL for better concurrency.
|
||||
- Put the`.sqlite`file outside webroot and set tight file permissions.
|
||||
|
||||
**CSV**
|
||||
|
||||
- Accept if data is: tiny, mostly read-only, and never queried/joined (e.g., a static list of countries).
|
||||
- Pros: human-readable, editable in Excel.
|
||||
- Cons: no indexes, no transactions, parsing edge cases, concurrency problems, security issues if stored in webroot.
|
||||
- If you use CSV writes, you must handle locking and validation yourself.
|
||||
|
||||
**NoSQL (MongoDB, CouchDB, etc.)**
|
||||
|
||||
- Use only if you already need document DB features (flexible schema, distributed clustering) or will scale horizontally soon.
|
||||
- Adds server/process to manage, drivers, extra operational complexity — usually overkill for low traffic.
|
||||
|
||||
**MySQL/Postgres**
|
||||
|
||||
- Good if you expect to scale or need multi-user DB server now; slightly more operational overhead than SQLite but easier to scale later.
|
||||
|
||||
# Practical considerations
|
||||
|
||||
- **Concurrency:**SQLite serializes writes. For low traffic or mostly-read sites this is fine. Turn on WAL mode (`PRAGMA journal_mode = WAL`) to improve concurrency.
|
||||
- **Backups:**SQLite = copy the file (use`VACUUM`or use`sqlite3`online backup API). CSV = copy file but risk partial writes. NoSQL = backup routines/replication needed.
|
||||
- **Security:**Never keep your DB/CSV in webroot. Correct file permissions. Use prepared statements to avoid injection.
|
||||
- **Querying & indexing:**CSV has none — searching/joins are O(n). SQLite supports indexes and complex queries.
|
||||
- **Migrations:**SQLite supports schema migration (use a simple migration script). CSV makes schema evolution painful.
|
||||
- **Tooling:**PHP has native PDO for SQLite. Many tools can open SQLite files. CSV editing via spreadsheets is easy but risky.
|
||||
|
||||
# Minimal PHP examples
|
||||
|
||||
**SQLite (PDO) — recommended**
|
||||
```
|
||||
PHP// open$pdo = new PDO('sqlite:/path/outside_webroot/app.db');$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);// create$pdo->exec("CREATE TABLE IF NOT EXISTS items (id INTEGER PRIMARY KEY, name TEXT, created_at INTEGER)");// insert (prepared)$stmt = $pdo->prepare('INSERT INTO items (name, created_at) VALUES (:name, :ts)');$stmt->execute([':name' => 'Example', ':ts' => time()]);// queryforeach ($pdo->query('SELECT * FROM items ORDER BY id DESC') as $row) { var_dump($row);}
|
||||
```
|
||||
|
||||
Tip: after creating DB run`PRAGMA journal_mode = WAL;`once to set WAL mode if you expect concurrent reads.
|
||||
|
||||
**CSV (with locking) — only for very simple data**
|
||||
```
|
||||
PHP$csvFile = '/path/outside_webroot/data.csv';// readif (($h = fopen($csvFile, 'r')) !== false) { while (($row = fgetcsv($h)) !== false) { // process $row array } fclose($h);}// append safelyif (($h = fopen($csvFile, 'a')) !== false) { if (flock($h, LOCK_EX)) { fputcsv($h, ['id', 'value', time()]); fflush($h); flock($h, LOCK_UN); } fclose($h);}
|
||||
```
|
||||
|
||||
# Rule of thumb
|
||||
|
||||
- If you need queries, joins, indexes, transactions →**SQLite**.
|
||||
- If it’s a single tiny static table you occasionally open in Excel →**CSV**.
|
||||
- If you need scaling, replication, distributed features, or already use NoSQL → consider**NoSQL**(but expect extra ops work).
|
||||
|
||||
If you want, I can:
|
||||
|
||||
- give a small migration plan from CSV → SQLite, or
|
||||
- provide a tiny PDO wrapper and migration scripts for your PHP project.
|
||||
|
||||
Which would you like next?
|
||||
|
||||
### User
|
||||
|
||||
this is basically to make a libary for theses of a school. how could I implement a export import of the sqlite to csv ?
|
||||
|
||||
### Assistant
|
||||
|
||||
Nice — good fit for SQLite. Below are ready-to-use options (PHP functions + CLI commands) to**export**and**import**CSVs safely and efficiently for your school theses library. I include streaming (no huge memory use), header handling, transactions, column mapping, basic validation, and tips for Excel compatibility.
|
||||
|
||||
# Quick notes before code
|
||||
|
||||
- Put DB file outside webroot and set tight permissions.
|
||||
- Always back up DB before imports.
|
||||
- Use transactions for imports for speed and atomicity.
|
||||
- For Excel, prepend a UTF-8 BOM (`\xEF\xBB\xBF`) when exporting so Excel recognizes UTF-8.
|
||||
- For large imports, process in batches and use prepared statements.
|
||||
- Validate rows (required columns, types, length) to avoid corrupting DB.
|
||||
|
||||
# 1) Export a table to CSV (streaming, low memory)
|
||||
```
|
||||
PHP<?phpfunction exportTableToCsv(PDO $pdo, string $table, string $csvPath, bool $includeHeader = true, string $bom = "\xEF\xBB\xBF") { $stmt = $pdo->prepare("SELECT * FROM \"$table\""); $stmt->execute(); $out = fopen($csvPath, 'w'); if ($out === false) { throw new RuntimeException("Cannot open CSV file for writing: $csvPath"); } // Write BOM for Excel/Windows compatibility (optional) if ($bom !== '') { fwrite($out, $bom); } // Write header if ($includeHeader) { $columnCount = $stmt->columnCount(); $headers = []; for ($i = 0; $i < $columnCount; $i++) { $meta = $stmt->getColumnMeta($i); $headers[] = $meta['name'] ?? "col$i"; } fputcsv($out, $headers); } // Stream rows while ($row = $stmt->fetch(PDO::FETCH_ASSOC)) { fputcsv($out, array_values($row)); } fclose($out);}
|
||||
```
|
||||
|
||||
Usage:
|
||||
```
|
||||
PHP$pdo = new PDO('sqlite:/path/outside_webroot/library.db');$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);exportTableToCsv($pdo, 'theses', '/tmp/theses_export.csv');
|
||||
```
|
||||
|
||||
# 2) Export an arbitrary query to CSV
|
||||
```
|
||||
PHPfunction exportQueryToCsv(PDO $pdo, string $sql, array $params, string $csvPath, bool $includeHeader = true, string $bom = "\xEF\xBB\xBF") { $stmt = $pdo->prepare($sql); $stmt->execute($params); $out = fopen($csvPath, 'w'); if ($out === false) throw new RuntimeException("Cannot open CSV file: $csvPath"); if ($bom !== '') fwrite($out, $bom); if ($includeHeader) { $columnCount = $stmt->columnCount(); $headers = []; for ($i = 0; $i < $columnCount; $i++) { $meta = $stmt->getColumnMeta($i); $headers[] = $meta['name'] ?? "col$i"; } fputcsv($out, $headers); } while ($row = $stmt->fetch(PDO::FETCH_NUM)) { fputcsv($out, $row); } fclose($out);}
|
||||
```
|
||||
|
||||
# 3) Import CSV into a table (stream + prepared insert, with header mapping)
|
||||
```
|
||||
PHP<?phpfunction importCsvToTable(PDO $pdo, string $csvPath, string $table, array $columnMap = null, int $batchSize = 500, string $encoding = 'UTF-8') { if (!file_exists($csvPath)) { throw new RuntimeException("CSV file not found: $csvPath"); } $h = fopen($csvPath, 'r'); if ($h === false) throw new RuntimeException("Cannot open CSV: $csvPath"); // Read header row $header = fgetcsv($h); if ($header === false) { fclose($h); throw new RuntimeException("CSV is empty or unreadable"); } // Optional: strip BOM from first header cell $header[0] = preg_replace('/^\x{FEFF}/u', '', $header[0]); // Map incoming CSV columns to DB columns. // $columnMap can be: null (assume header names match table columns) // or an associative array: ['CSV Column Name' => 'db_column'] $map = []; if ($columnMap === null) { foreach ($header as $col) $map[$col] = $col; } else { foreach ($header as $col) { if (isset($columnMap[$col])) $map[$col] = $columnMap[$col]; else $map[$col] = null; // skip if not mapped } } // Determine DB columns we will insert (in order) $dbCols = array_values(array_filter($map, fn($v) => $v !== null)); if (count($dbCols) === 0) { fclose($h); throw new RuntimeException("No CSV columns map to DB columns."); } // Prepare insert statement $placeholders = implode(',', array_fill(0, count($dbCols), '?')); $colList = implode(',', array_map(fn($c) => "\"$c\"", $dbCols)); $insertSql = "INSERT INTO \"$table\" ($colList) VALUES ($placeholders)"; $insertStmt = $pdo->prepare($insertSql); // Read rows in batches inside a transaction $rowCount = 0; $batch = []; $pdo->beginTransaction(); try { while (($row = fgetcsv($h)) !== false) { $assoc = array_combine($header, $row); // Build values in dbCols order; do simple validation (you can extend) $values = []; foreach ($map as $csvCol => $dbCol) { if ($dbCol === null) continue; } foreach ($dbCols as $dbCol) { // find the CSV column that maps to this dbCol (reverse lookup) $csvKey = array_search($dbCol, $map, true); $values[] = $assoc[$csvKey] ?? null; } // Optional: basic validation example (skip blank required rows) $allEmpty = true; foreach ($values as $v) { if (trim((string)$v) !== '') { $allEmpty = false; break; } } if ($allEmpty) continue; $insertStmt->execute($values); $rowCount++; if ($rowCount % $batchSize === 0) { $pdo->commit(); $pdo->beginTransaction(); } } $pdo->commit(); } catch (Exception $e) { $pdo->rollBack(); fclose($h); throw $e; } fclose($h); return $rowCount;}
|
||||
```
|
||||
|
||||
Usage with column mapping:
|
||||
```
|
||||
PHP$colMap = [ 'Title' => 'title', 'Author' => 'author', 'Year' => 'year', 'Supervisor' => 'supervisor'];$count = importCsvToTable($pdo, '/tmp/theses_import.csv', 'theses', $colMap);echo "Imported $count rows\n";
|
||||
```
|
||||
|
||||
# 4) Tips & extras
|
||||
|
||||
**Detect table columns automatically**
|
||||
```
|
||||
PHPfunction getTableColumns(PDO $pdo, string $table): array { $cols = []; $stmt = $pdo->query("PRAGMA table_info(\"$table\")"); foreach ($stmt->fetchAll(PDO::FETCH_ASSOC) as $col) { $cols[] = $col['name']; } return $cols;}
|
||||
```
|
||||
|
||||
**Speed optimizations for large imports**
|
||||
|
||||
- Wrap inserts in a transaction (done above).
|
||||
- Turn off synchronous temporarily for import:`PRAGMA synchronous = OFF;`then set back to`NORMAL`or`FULL`afterwards. (Be careful — power loss can cause corruption during import.)
|
||||
- Use`PRAGMA journal_mode = WAL;`for better concurrency.
|
||||
|
||||
**CSV dialect**
|
||||
|
||||
- Use`fgetcsv`/`fputcsv`which handle quoting/commas.
|
||||
- If you must support Excel-generated CSVs, handle`\r\n`line endings and possibly semicolon delimiters (region dependent).
|
||||
|
||||
**Conflict handling**
|
||||
|
||||
- If`theses`table has unique constraints (e.g.,`unique(identifier)`), you can use`INSERT OR REPLACE`or`INSERT OR IGNORE`depending on desired behavior:```
|
||||
SQLINSERT OR REPLACE INTO "theses" (col1, col2) VALUES (?, ?)
|
||||
```
|
||||
|
||||
**CSV created by CLI (SQLite)**
|
||||
|
||||
- If you have shell access, you can quickly export:```
|
||||
Bashsqlite3 library.db -header -csv "SELECT * FROM theses;" > theses.csv
|
||||
```
|
||||
|
||||
- Import via sqlite3 CLI (beware: requires exact columns & table prepared):```
|
||||
Bashsqlite3 library.db.mode csv.import /path/to/theses.csv theses
|
||||
```
|
||||
|
||||
# 5) Example end-to-end flow for your theses library
|
||||
|
||||
- Create`theses`schema in SQLite with fields:`id, title, author, year, department, supervisor, abstract, filepath, created_at`.
|
||||
- Provide an admin page to**Export**: call`exportTableToCsv()`and stream the CSV to browser with correct headers and BOM.
|
||||
- Provide an admin**Import**form (file upload). On upload:
|
||||
|
||||
- Save uploaded CSV to a temp folder outside webroot.
|
||||
- Make a backup copy of DB.
|
||||
- Run`importCsvToTable()`with a column map (allow admins to map CSV columns to DB fields if header names differ).
|
||||
- Show import summary: rows imported, rows skipped, errors.
|
||||
- For repeatable imports, keep a small import-log table to record timestamp, filename, rows imported, and admin user.
|
||||
|
||||
If you want, I can:
|
||||
|
||||
- produce a full single-file PHP admin script (upload UI + export button + import with mapping UI + summary) tailored to your`theses`schema, or
|
||||
- give a tiny migration script to convert an existing CSV folder into the`theses`SQLite DB.
|
||||
|
||||
Which would you like me to generate now?
|
||||
|
||||
Reference in New Issue
Block a user