On this page

Troubleshooting

Symptoms by area, with the cause that most often explains them.

Sign-in and identity#

SymptomLikely cause
Every API call returns 401 after loginIssuer or JWKS mismatch — compare KEYCLOAK_ISSUER with the realm, and confirm the API can reach KEYCLOAK_JWKS_URL.
Login redirects foreverThe realm's valid redirect URIs do not match the deployment.
Safari bounces between app and loginThe session cookie exceeds 4 KB — set SESSION_STORE_DIR.
/admin returns 403The account lacks the admin realm role.
A role change has no effectRoles come from the access token; wait for the next refresh or sign in again.

Imports#

SymptomLikely cause
Import fails immediatelyUnsupported or corrupt file — read the job's error and re-export the source.
Modules report unavailable after a successful importColumn roles are wrong — remap them in the log settings.
Time bounds empty, trends flatThe timestamp column was misidentified or misparsed.
One giant case in a CSVThe case id column was mapped to a near-constant column.
Staged upload vanishedStaging is cleared after two hours — re-upload.
Upload refused with 413It trips a zip-bomb guard: overlapping archive entries, an implausible compression ratio, or more than INGEST_MAX_DECOMPRESSED_BYTES unpacked.
First import on a fresh host takes very longModule environments are being materialised; later imports are fast.

Jobs and modules#

SymptomLikely cause
Progress bar stalls with a "no progress" hintThe module reports nothing; cancel and retry, or fix the module.
Job fails with a timeoutIt exceeds JOB_EXECUTION_TIMEOUT_SECONDS.
A module fails to loadManifest, install, or import error — API log, then the module detail page's log tail.
Cards stay LimitedAn optional dependency is missing.
Installed module does not appearThe install job failed and rolled back.
Subprocess module restarts repeatedlyA crash at worker startup; the bridge logs the exit code and attempt count.

Deployment#

SymptomLikely cause
Proxy fails to start with TLS errorsOnly live/ was mounted, so the certificate symlinks into archive/ break — mount all of /etc/letsencrypt.
An .env change has no effectThe container was restarted, not recreated.
A realm change has no effectThe JSON imports only into an empty database.
Disk grows without bound with S3 configuredNo cache budget, or the reaper is still in dry-run.
Uploads fail with 507The bucket quota is reached.
Everything 500s after a deployA migration failed on boot — check docker compose logs api around the alembic step.
Job updates never arrive, the page works otherwiseSomething in the proxy chain buffers text/event-stream.

MCP#

SymptomLikely cause
503 "MCP is disabled"Boot flag off, or the live admin kill switch is on.
401 on every callToken missing, expired, revoked, or wrong OAuth audience.
[consent_required]The user has not enabled external data access.
[scope_missing]The token lacks the scope; mint a new one.
[read_only] on a write toolServer-wide read-only.
[conflict] "… is computing for this process"The read started the module; wait_for_job on the job id in the message, then repeat the call.
[conflict] "… failed for this process"The module's run failed; the message carries its error (CV4CDD, for example, needs a model).
404 on /mcpThe flag never reached the container, or the proxy route is missing.
Output truncatedThe 200 KB per-result cap — narrow the query or paginate.

Performance#

SymptomLikely cause
Dashboards slow to paintMany cards, each querying DuckDB — reduce cards, cap DUCKDB_THREADS.
One module slow only on large logsRow-by-row Python instead of SQL.
API restarts under loadMemory pressure from a module — isolate it, or bound its allocations.
Everyone waits for one user's jobsThe offload pool is saturated — set MAX_OFFLOADS_PER_USER.

Diagnosing an unknown failure#

  1. Reproduce with a synthetic log of 200 cases — most module bugs show up immediately.

  2. Collect Settings → About → Copy diagnostics and the failing job's log tail.

  3. Read the module's raw manifest: GET/api/v1/modules/{id}/manifest shows what the platform actually parsed.

  4. Search the API log for the log id — ingest and module runs log both. GET/api/v1/event-logs/{id}/module-runs shows each module's state for the log.

  5. Still stuck? Open an issue with the diagnostics blob, the steps, and the version: github.com/Process-Science-Community/MATE/issues.