hrtm-benko

Jakarta EE / TomEE application with an Angular frontend for analysing Austrian municipal finance data.

PostgreSQL

The application expects a TomEE JTA datasource named jdbc/hrtmBenko.

Example TomEE resource:

<Resource id="jdbc/hrtmBenko" type="DataSource">
  JdbcDriver = org.postgresql.Driver
  JdbcUrl = jdbc:postgresql://POSTGRES_LXC:5432/hrtm_benko
  UserName = hrtm_benko
  Password = hrtm_benko
  JtaManaged = true
</Resource>

Create the first import table with:

psql "$DATABASE_URL" -f sql/001_schema.sql

Build

Backend only:

mvn clean verify

WAR with Angular frontend:

mvn clean package -Pfrontend

Documentation

The packaged WAR includes a static documentation page at /docs/. In production with the /hrtm-benko context path, it is available at /hrtm-benko/docs/ and linked from the public analysis page.

Tests

mvn clean verify -Pfrontend runs the backend tests, frontend Vitest suite, Playwright browser tests, production frontend build and WAR packaging. After deployment, Forgejo additionally runs:

frontend/node/node scripts/full-stack-smoke.mjs

The full-stack smoke suite uses the deployed TomEE application and its configured PostgreSQL datasource. It dynamically selects an existing dataset for a real read-only analysis, executes SELECT 1 through the protected admin query endpoint, verifies database readiness and deployed build metadata, checks the public reverse-proxy boundary and confirms that legacy or unprotected write requests are rejected before they can mutate data. Base URLs and retry timings can be overridden through SMOKE_INTERNAL_BASE_URL, SMOKE_PUBLIC_BASE_URL, SMOKE_RETRY_TIMEOUT_MS and SMOKE_RETRY_INTERVAL_MS. CI also sets SMOKE_EXPECTED_COMMIT, so the smoke test waits for and verifies the exact deployment produced by the current run.

Operations diagnostics

• GET /api/health is a readiness endpoint. It executes SELECT 1 through the configured TomEE datasource with a two-second query timeout. A healthy database returns HTTP 200 with status=ok, database=ok and databaseDurationMs; an unavailable datasource or database returns HTTP 503 without exposing connection details.
• GET /api/version returns the application name and version, deployed commit SHA, UTC build time and application start time. Forgejo injects commit and build time into the packaged WAR.
• Production EclipseLink logging is limited to warnings and does not log SQL parameter values.

Shareable analysis URLs

Applying analysis filters writes a canonical URL containing the selected municipalities, periods or years, budget, visible metrics, display mode, account-only mode, investment-project visibility and MVAG prefixes. Browser back/forward restores the encoded state. The analysis toolbar offers a share button that uses the Web Share API when available and otherwise copies the URL to the clipboard.

Import

The public Angular application is read-only. Imports are handled by a separate admin page:

• Admin UI: /admin/import.html
• Dashboard configuration UI: /admin/configuration.html
• Read-only SQL UI: /admin/query.html
• Budget CSV upload endpoint: POST /api/imports/csv
• Transparenzportal budget HTML URL endpoint: POST /api/imports/budget-html
• Municipality revenue-sharing or population XLSX upload endpoint: POST /api/imports/municipality-revenue-sharing
• Source PDF upload endpoint: POST /api/admin/finance/source-documents
• Duplicate cleanup endpoint: POST /api/admin/finance/datasets/duplicates/cleanup
• Public source PDF download endpoint: GET /api/finance/source-documents
• Multipart field: file
• Optional form field: truncate=true

The HTML import accepts sourceUrl=https://transparenz.hrtm.net/council-manager/resources/finance/council/.../*.html. HTML rows are treated as fallback data: the import replaces only existing HTML fallback rows for the same municipality/year/report type/household, refuses to overwrite primary CSV rows, and later CSV imports remove matching HTML fallback rows.

Protect /admin/*, /api/imports/* and /api/admin/* at the reverse proxy. Hiding the admin pages alone is not sufficient because the endpoints write data. All mutating requests below /api/imports/* and /api/admin/* additionally require the header X-HRTM-Benko-Admin: 1; cross-site browser requests and mismatching Origin headers are rejected. This is a CSRF defense in depth and does not replace the reverse-proxy access restriction.

The SQL endpoint is fail-closed even when the page is reachable. Enable it only after the admin paths are protected by setting HRTM_BENKO_ADMIN_QUERY_ENABLED=true (or the JVM property -Dhrtm.benko.admin.query.enabled=true). It accepts one SELECT/WITH statement, runs it in a database read-only transaction, enforces a five-second timeout and caps rows, columns, individual cells and total response size. The UI offers row limits of 10, 100 and 1000; the server always enforces the selected cap independently of SQL text sent by the browser.

Example Caddy rule:

@benko_import path /hrtm-benko/admin/* /hrtm-benko/api/imports/* /hrtm-benko/api/admin/*
basicauth @benko_import {
    admin $2a$14$REPLACE_WITH_HASHED_PASSWORD
}

Example nginx location:

location ~ ^/hrtm-benko/(admin/|api/imports/|api/admin/) {
    auth_basic "HRTM Benko Import";
    auth_basic_user_file /etc/nginx/htpasswd-hrtm-benko-import;
    proxy_pass http://tomee_backend;
}