Troubleshooting
Symptoms by area, with the cause that most often explains them.
Sign-in and identity#
| Symptom | Likely cause |
|---|---|
Every API call returns 401 after login | Issuer or JWKS mismatch — compare KEYCLOAK_ISSUER with the realm, and confirm the API can reach KEYCLOAK_JWKS_URL. |
| Login redirects forever | The realm's valid redirect URIs do not match the deployment. |
| Safari bounces between app and login | The session cookie exceeds 4 KB — set SESSION_STORE_DIR. |
/admin returns 403 | The account lacks the admin realm role. |
| A role change has no effect | Roles come from the access token; wait for the next refresh or sign in again. |
Imports#
| Symptom | Likely cause |
|---|---|
| Import fails immediately | Unsupported or corrupt file — read the job's error and re-export the source. |
| Modules report unavailable after a successful import | Column roles are wrong — remap them in the log settings. |
| Time bounds empty, trends flat | The timestamp column was misidentified or misparsed. |
| One giant case in a CSV | The case id column was mapped to a near-constant column. |
| Staged upload vanished | Staging is cleared after two hours — re-upload. |
Upload refused with 413 | It trips a zip-bomb guard: overlapping archive entries, an implausible compression ratio, or more than INGEST_MAX_DECOMPRESSED_BYTES unpacked. |
| First import on a fresh host takes very long | Module environments are being materialised; later imports are fast. |
Jobs and modules#
| Symptom | Likely cause |
|---|---|
| Progress bar stalls with a "no progress" hint | The module reports nothing; cancel and retry, or fix the module. |
| Job fails with a timeout | It exceeds JOB_EXECUTION_TIMEOUT_SECONDS. |
| A module fails to load | Manifest, install, or import error — API log, then the module detail page's log tail. |
| Cards stay Limited | An optional dependency is missing. |
| Installed module does not appear | The install job failed and rolled back. |
| Subprocess module restarts repeatedly | A crash at worker startup; the bridge logs the exit code and attempt count. |
Deployment#
| Symptom | Likely cause |
|---|---|
| Proxy fails to start with TLS errors | Only live/ was mounted, so the certificate symlinks into archive/ break — mount all of /etc/letsencrypt. |
An .env change has no effect | The container was restarted, not recreated. |
| A realm change has no effect | The JSON imports only into an empty database. |
| Disk grows without bound with S3 configured | No cache budget, or the reaper is still in dry-run. |
Uploads fail with 507 | The bucket quota is reached. |
| Everything 500s after a deploy | A migration failed on boot — check docker compose logs api around the alembic step. |
| Job updates never arrive, the page works otherwise | Something in the proxy chain buffers text/event-stream. |
MCP#
| Symptom | Likely cause |
|---|---|
503 "MCP is disabled" | Boot flag off, or the live admin kill switch is on. |
401 on every call | Token missing, expired, revoked, or wrong OAuth audience. |
[consent_required] | The user has not enabled external data access. |
[scope_missing] | The token lacks the scope; mint a new one. |
[read_only] on a write tool | Server-wide read-only. |
[conflict] "… is computing for this process" | The read started the module; wait_for_job on the job id in the message, then repeat the call. |
[conflict] "… failed for this process" | The module's run failed; the message carries its error (CV4CDD, for example, needs a model). |
404 on /mcp | The flag never reached the container, or the proxy route is missing. |
| Output truncated | The 200 KB per-result cap — narrow the query or paginate. |
Performance#
| Symptom | Likely cause |
|---|---|
| Dashboards slow to paint | Many cards, each querying DuckDB — reduce cards, cap DUCKDB_THREADS. |
| One module slow only on large logs | Row-by-row Python instead of SQL. |
| API restarts under load | Memory pressure from a module — isolate it, or bound its allocations. |
| Everyone waits for one user's jobs | The offload pool is saturated — set MAX_OFFLOADS_PER_USER. |
Diagnosing an unknown failure#
Reproduce with a synthetic log of 200 cases — most module bugs show up immediately.
Collect Settings → About → Copy diagnostics and the failing job's log tail.
Read the module's raw manifest: GET
/api/v1/modules/{id}/manifestshows what the platform actually parsed.Search the API log for the log id — ingest and module runs log both. GET
/api/v1/event-logs/{id}/module-runsshows each module's state for the log.Still stuck? Open an issue with the diagnostics blob, the steps, and the version: github.com/Process-Science-Community/MATE/issues.