Configuration reference
Every setting, with its default, as in config/lorebase.example.yaml. The services read one YAML file (LOREBASE_CONFIG), and an environment variable LOREBASE__SECTION__KEY overrides the key section.key; nested keys use more double underscores. Every key is optional.
Lorebase example configuration.
Point LOREBASE_CONFIG at a copy of this file. Precedence, lowest to highest:
code defaults < this file < environment variables LOREBASE__SECTION__KEY
(nested keys use more double underscores: LOREBASE__INFERENCE__LLM__BASE_URL).
The values below are the defaults compiled into the code; every key is optional.
Secret settings accept a literal, `secret://env/NAME` (environment variable) or
`secret://file//absolute/path` (file content).
logging
| Setting |
Default |
Description |
logging.level |
INFO |
CRITICAL | ERROR | WARNING | INFO | DEBUG |
logging.format |
json |
json | text |
logging.redact_keys |
[password, secret, token, api_key, authorization, credential] |
field names whose values are masked in every log record |
logging.redact_min_length |
4 |
shorter secret values are not masked inside free text |
telemetry
| Setting |
Default |
Description |
telemetry.otlp_endpoint |
null |
e.g. http://otel-collector:4317; unset disables telemetry |
telemetry.service_name |
lorebase |
|
telemetry.service_namespace |
lorebase |
|
telemetry.sample_ratio |
0.1 |
fraction of root traces sampled, 0 to 1 |
telemetry.metric_export_interval_s |
60 |
|
keeper
Keeper, the REST API service. Environment: LOREBASE__KEEPER__PORT, LOREBASE__KEEPER__HOST, ...
| Setting |
Default |
Description |
keeper.host |
127.0.0.1 |
bind address; containers set 0.0.0.0 |
keeper.port |
8080 |
|
keeper.api_prefix |
/api/v1 |
version prefix of every API route; OpenAPI at <prefix>/openapi.json |
keeper.request_id_header |
X-Request-ID |
accepted from clients when well formed, echoed back |
keeper.problem_type_base |
"urn:lorebase:problem:" |
error type URI = this prefix + error code |
keeper.trusted_proxies |
[] |
networks of reverse proxies in front of Keeper, e.g. [10.0.0.0/8]. Only from these peers is X-Forwarded-For read (right to left, trusted hops skipped) to find the client address used by API key allowlists; empty: the socket peer is the client |
| Setting |
Default |
Description |
keeper.security_headers.enabled |
true |
master switch for the response headers below |
| Setting |
Default |
Description |
keeper.security_headers.hsts.enabled |
auto |
Strict-Transport-Security: auto (only when the request came over https, directly or from a trusted proxy sending X-Forwarded-Proto: https), always, never |
keeper.security_headers.hsts.max_age_s |
31536000 |
|
keeper.security_headers.hsts.include_subdomains |
true |
|
keeper.security_headers.hsts.preload |
false |
needs include_subdomains and max_age_s >= 31536000 |
| Setting |
Default |
Description |
keeper.security_headers.frame_options |
DENY |
X-Frame-Options and CSP frame-ancestors: DENY, SAMEORIGIN or off |
keeper.security_headers.referrer_policy |
no-referrer |
Referrer-Policy; empty: not sent |
keeper.security_headers.content_type_nosniff |
true |
X-Content-Type-Options: nosniff |
keeper.cors
| Setting |
Default |
Description |
keeper.cors.allow_origins |
[] |
browser origins allowed to call the API; empty disables CORS |
keeper.cors.allow_methods |
[GET, POST, PUT, PATCH, DELETE, OPTIONS] |
|
keeper.cors.allow_headers |
[] |
further request headers; Authorization, Content-Type, Idempotency-Key, If-None-Match, the request id, CSRF and trace headers are always allowed |
keeper.cors.expose_headers |
[ETag, Lorebase-Revision] |
readable by scripts (always: these, the request id and the rate limit headers) |
keeper.cors.allow_credentials |
false |
cannot be combined with the "*" origin |
keeper.cors.max_age_s |
600 |
|
keeper.limits
| Setting |
Default |
Description |
keeper.limits.max_request_body_bytes |
1048576 |
larger bodies are rejected with 413 |
keeper.limits.rate
Token buckets per principal (API key id or user) and request class; refusals are 429 with Retry-After, every response of a limited route carries RateLimit-Limit/-Remaining/-Reset
| Setting |
Default |
Description |
keeper.limits.rate.enabled |
true |
|
keeper.limits.rate.backend |
local |
local (alias memory): per process, N Keepers allow N times the limits; postgres: buckets shared by every Keeper (use it with more than one; compose scale-out and Helm with replicas > 1) |
keeper.limits.rate.postgres |
|
the shared backend: UNLOGGED table rate_limit_buckets |
keeper.limits.rate.postgres.lease_tokens |
10 |
tokens taken per query and served from memory (at most a quarter of a bucket); unused ones expire, never overspent |
keeper.limits.rate.postgres.lease_ttl_s |
2.0 |
leased tokens not used in this time are dropped |
keeper.limits.rate.postgres.timeout_s |
0.25 |
budget of one query; slower counts as a failure |
keeper.limits.rate.postgres.on_error |
local |
database down or slow: local (limits in this process), allow or deny; failed-authentication buckets always deny |
keeper.limits.rate.postgres.error_retry_after_s |
1 |
Retry-After of a refusal caused by the database |
keeper.limits.rate.postgres.check_cache_s |
0.5 |
reuse of a failed-authentication bucket reading by the check before each authentication (failures are written at once) |
keeper.limits.rate.postgres.auth_failures_timeout_s |
1.0 |
budget of a failed-authentication query; then 429 |
keeper.limits.rate.postgres.stale_reading_s |
30.0 |
during an outage a reading this recent still decides, so only clients without one are refused |
keeper.limits.rate.postgres.prune_interval_s |
60.0 |
deletion of rows of buckets that are full again |
keeper.limits.rate.postgres.prune_batch |
1000 |
|
keeper.limits.rate.max_buckets |
100000 |
buckets kept in memory; least recently used are dropped |
keeper.limits.rate.exempt_paths |
[/healthz, /readyz] |
route paths without the API prefix, never limited |
keeper.limits.rate.reads |
|
GETs, query, traverse, neighborhood |
keeper.limits.rate.reads.requests_per_minute |
1200 |
sustained rate |
keeper.limits.rate.reads.burst |
200 |
requests allowed back to back |
keeper.limits.rate.writes |
|
every other POST, PUT, PATCH and DELETE |
keeper.limits.rate.writes.requests_per_minute |
300 |
|
keeper.limits.rate.writes.burst |
60 |
|
keeper.limits.rate.searches |
|
POST .../search |
keeper.limits.rate.searches.requests_per_minute |
300 |
|
keeper.limits.rate.searches.burst |
60 |
|
keeper.limits.rate.context |
|
POST .../context (context packs) |
keeper.limits.rate.context.requests_per_minute |
120 |
|
keeper.limits.rate.context.burst |
30 |
|
keeper.limits.rate.admin |
|
routes tagged admin: keys, principals, grants, audit |
keeper.limits.rate.admin.requests_per_minute |
120 |
|
keeper.limits.rate.admin.burst |
30 |
|
keeper.limits.rate.auth_failures |
|
401s to requests carrying a credential, per client address |
keeper.limits.rate.auth_failures.requests_per_minute |
30 |
and per API key prefix; when either is spent, requests are |
keeper.limits.rate.auth_failures.burst |
30 |
answered 429 before the credential is checked |
keeper.limits.rate.auth_failures.ipv6_prefix_length |
64 |
IPv6 clients are counted by network of this length |
keeper.limits.rate.auth_failures.per_key_prefix_enabled |
true |
false: count by address only (key prefixes are not secret, so with a bucket per prefix whoever knows one can keep that key throttled); the per-address bucket is always on |
keeper.limits.rate.domains |
{} |
per-Domain buckets by Domain id, any of the classes above, e.g. supply: {writes: {requests_per_minute: 60, burst: 10}} |
keeper.limits.rate.routes |
|
how requests are sorted into classes |
keeper.limits.rate.routes.admin_tags |
[admin] |
|
keeper.limits.rate.routes.search_suffixes |
[/search] |
|
keeper.limits.rate.routes.context_suffixes |
[/context] |
|
keeper.limits.rate.routes.read_post_suffixes |
[/query, /traverse, /neighborhood] |
|
keeper.limits.rate.throttle_audit |
|
audit entry rate_limit.throttled for a principal that keeps |
keeper.limits.rate.throttle_audit.threshold |
20 |
being refused: this many refusals within window_s; 0 disables |
keeper.limits.rate.throttle_audit.window_s |
300 |
|
keeper.limits.rate.throttle_audit.max_tracked |
10000 |
|
keeper.limits.quota
Daily quotas per API key, in UTC days (users have none); exceeded: 429 quota-exceeded with Retry-After to the reset. Usage is stored in api_key_usage and shown by GET /auth/whoami
| Setting |
Default |
Description |
keeper.limits.quota.enabled |
true |
|
keeper.limits.quota.requests_per_day |
100000 |
null: unlimited |
keeper.limits.quota.context_tokens_per_day |
2000000 |
estimated tokens of context packs; null: unlimited |
keeper.limits.quota.flush_interval_s |
5 |
a process writes its counted use this often |
keeper.limits.quota.keys |
{} |
per API key id, e.g. key_01...: {requests_per_day: null} |
| Setting |
Default |
Description |
keeper.pagination.default_limit |
50 |
|
keeper.pagination.max_limit |
500 |
|
keeper.reads
| Setting |
Default |
Description |
keeper.reads.reachability_max_depth |
1000 |
parent hops followed to prove that a Revision or tag named by a caller with branch-scoped grants is reachable from a branch the caller may read; further back: 403 |
keeper.reads.cache
In-process cache of GET read responses, keyed by Revision:
| Setting |
Default |
Description |
keeper.reads.cache.enabled |
true |
commits move branches to new Revisions, so no invalidation |
keeper.reads.cache.max_entries |
1024 |
|
keeper.reads.cache.max_bytes |
33554432 |
32 MiB of response bodies |
keeper.reads.cache.ttl_s |
300 |
|
keeper.reads.cache.max_body_bytes |
1048576 |
larger responses are not cached |
keeper.reads.cache.time_travel_max_entries |
128 |
entries for reads at another Revision; 0 never caches them |
keeper.reads.sensitivity
Values of pii / confidential / restricted Attributes, shown only with pii.read / confidential.read / restricted.read
| Setting |
Default |
Description |
keeper.reads.sensitivity.omit |
false |
true: leave masked values out of Records |
keeper.reads.sensitivity.masked_value |
"***" |
what masked values read as |
keeper.reads.sensitivity.erased_value |
"[erased]" |
what erased vault values read as (cleared callers) |
keeper.reads (continued)
| Setting |
Default |
Description |
keeper.reads.prefer_replica |
false |
true: read endpoints use the replicas of database.read_dsn when one is fresh enough (see docs/deployment.md) |
keeper.readiness
| Setting |
Default |
Description |
keeper.readiness.timeout_s |
2 |
time budget of each /readyz check |
keeper.readiness.database_section |
database |
settings section with the connection string; if absent the |
keeper.readiness.database_dsn_key |
dsn |
database check reports not_configured |
keeper.auth
| Setting |
Default |
Description |
keeper.auth.mode |
none |
none | api_key | oidc | api_key+oidc. none accepts every request as an administrator: local development only. The other modes need the database section below |
keeper.auth.allow_insecure |
false |
true: allow mode none with a host other than loopback; otherwise Keeper refuses to start with that combination |
keeper.auth.installation_admins |
[] |
principal ids, identity provider subjects or group ids allowed to administer the installation (list principals). They get no access to Domain data |
keeper.auth.api_keys
Agents and automation: Authorization: Bearer lbk_... or X-Api-Key
| Setting |
Default |
Description |
keeper.auth.api_keys.token_prefix |
lbk |
keys look like <token_prefix><prefix><secret> |
keeper.auth.api_keys.header |
X-Api-Key |
header accepted besides Authorization: Bearer |
keeper.auth.api_keys.pepper |
null |
REQUIRED with api_key modes: >= 32 characters, e.g. secret://env/LOREBASE_API_KEY_PEPPER. Changing it invalidates every issued key |
keeper.auth.api_keys.prefix_length |
10 |
characters of the public lookup prefix |
keeper.auth.api_keys.secret_bytes |
32 |
random bytes of the secret part |
keeper.auth.api_keys.default_ttl_days |
365 |
lifetime of new keys; null: no expiry |
keeper.auth.api_keys.max_ttl_days |
null |
longest lifetime an admin may request; null: no limit |
keeper.auth.api_keys.last_used_interval_s |
60 |
a key's last-used time is written at most this often |
keeper.auth.api_keys.max_allowed_networks |
32 |
most client networks a key's allowed_networks may list |
keeper.auth.oidc
Bearer tokens of any OpenID Connect provider
| Setting |
Default |
Description |
keeper.auth.oidc.issuer |
null |
REQUIRED with oidc modes: expected iss, base of discovery |
keeper.auth.oidc.audience |
[] |
REQUIRED with oidc modes: accepted aud values |
keeper.auth.oidc.jwks_uri |
null |
null: read from <issuer>/.well-known/openid-configuration |
keeper.auth.oidc.algorithms |
[RS256, ES256] |
asymmetric algorithms only |
keeper.auth.oidc.clock_skew_s |
60 |
tolerance for exp, nbf and iat |
keeper.auth.oidc.jwks_cache_ttl_s |
300 |
signing keys are refreshed after this long |
keeper.auth.oidc.jwks_min_refresh_s |
30 |
shortest time between key refreshes (unknown key id, errors) |
keeper.auth.oidc.http_timeout_s |
5 |
timeout of provider requests |
keeper.auth.oidc.subject_claim |
sub |
|
keeper.auth.oidc.name_claims |
[name, preferred_username, email] |
display name, in order of preference |
keeper.auth.oidc.groups_claim |
groups |
claim with provider group names (matched to groups) |
keeper.auth.oidc.auto_provision |
true |
create the user on the first valid token |
keeper.auth.oidc.principal_cache_ttl_s |
60 |
reuse of a subject's principal and groups |
keeper.auth.oidc.principal_cache_max_entries |
4096 |
|
keeper.auth.oidc.allow_insecure |
false |
true: accept http issuer and jwks_uri (configured or discovered) off loopback; https is required otherwise |
keeper.auth.authz
| Setting |
Default |
Description |
keeper.auth.authz.grants_cache_ttl_s |
15 |
grants are cached this long; other processes see changes after at most this; 0 disables |
keeper.auth.authz.grants_cache_max_entries |
2048 |
|
keeper.auth.authz.invisible_kinds |
true |
Kinds a caller's Kind-narrowed grants leave out are invisible to it: the schema it reads omits them and what tells about them, and naming them answers like an unknown name (404/422, never 403); false refuses them with 403 naming the Kind |
keeper.auth.authz.custom_roles |
{} |
extra roles, name -> permissions. Built-in roles: viewer, explorer, editor, operator, modeler, reviewer, maintainer, admin, agent-reader. Example: contributor: [schema.read, data.read, branch.create] Permissions: schema.read data.read views.write branch.create operation.execute operation.execute_custom schema.write source.map proposal.review conflict.resolve proposal.merge source.manage job.manage grant.manage apikey.manage domain.settings domain.delete audit.read trace.read trace.explain principal.read domain.create (the last two are installation-level) |
keeper.auth.audit
| Setting |
Default |
Description |
keeper.auth.audit.enabled |
true |
hash-chained log of mutations and denied requests; needs the database section |
keeper.auth.audit.record_denied |
true |
also record requests rejected with 401 or 403 |
keeper.auth.audit.write_timeout_s |
2 |
a failed or slow write is logged and counted, never surfaced |
keeper.auth.audit.reads |
|
records of successful reads, in audit_read_log (hash chains of their own), written in batches off the request path |
keeper.auth.audit.reads.enabled |
false |
|
keeper.auth.audit.reads.sample_rate |
1.0 |
share of the reads passing the filters that are recorded |
keeper.auth.audit.reads.principal_types |
[] |
user, service, api_key; empty: every type |
keeper.auth.audit.reads.kinds |
[] |
Kind api names in the route; empty: every read reads that showed or masked a sensitive value are always recorded, whatever the sampling and the filters |
keeper.auth.audit.reads.queue_size |
10000 |
past it records are dropped (lorebase.audit.reads.dropped) |
keeper.auth.audit.reads.batch_size |
500 |
records per transaction |
keeper.auth.audit.reads.flush_interval_s |
1.0 |
longest wait before queued records are written |
keeper.auth.audit.reads.write_timeout_s |
5.0 |
a failed batch is dropped and counted (on_overflow: drop) |
keeper.auth.audit.reads.on_overflow |
drop |
drop: serve the read, drop and count the record; reject: 503 when the record finds no place in the queue, failed batches kept and retried (the read stream as proof) |
keeper.auth.audit.reads.retry_after_s |
1 |
Retry-After of reads refused by on_overflow: reject |
keeper.auth.audit.reads.unready_after_failures |
3 |
failed attempts in a row of a batch before readiness fails (on_overflow: reject) |
keeper.auth.sessions
Sign-in of the web app: Keeper as backend-for-frontend (authorization code + PKCE, server-side sessions). Needs an oidc mode and the database section
| Setting |
Default |
Description |
keeper.auth.sessions.enabled |
false |
|
keeper.auth.sessions.discovery_url |
null |
null: <oidc.issuer>/.well-known/openid-configuration |
keeper.auth.sessions.client_id |
null |
REQUIRED when enabled |
keeper.auth.sessions.client_secret |
null |
e.g. secret://env/LOREBASE_OIDC_CLIENT_SECRET; not needed with token_endpoint_auth_method: none (public client, PKCE only) |
keeper.auth.sessions.token_endpoint_auth_method |
client_secret_basic |
| client_secret_post | none |
keeper.auth.sessions.scopes |
[openid, profile, email] |
add offline_access if the provider needs it for refresh tokens |
keeper.auth.sessions.redirect_uri |
null |
REQUIRED: public URL of <api_prefix>/auth/callback, registered at the provider and sent unchanged (exact match) |
keeper.auth.sessions.app_url |
null |
REQUIRED: public URL of the web app; sign-in returns to a path on its origin |
keeper.auth.sessions.post_logout_redirect_uri |
null |
after sign-out at the provider; null: app_url |
keeper.auth.sessions.error_path |
/login |
app path receiving failed sign-ins as ?error=<code> |
keeper.auth.sessions.encryption_key |
null |
REQUIRED: >= 32 characters, e.g. secret://env/LOREBASE_SESSION_KEY; encrypts the sign-in state cookie and stored provider tokens |
keeper.auth.sessions.cookie_name |
lorebase_session |
sent as __Host-<name> with cookie_secure |
keeper.auth.sessions.login_cookie_name |
lorebase_login |
short-lived sign-in state cookie |
keeper.auth.sessions.cookie_secure |
true |
Secure + __Host- prefix; false only for http on loopback |
keeper.auth.sessions.same_site |
lax |
lax | strict (session cookie) |
keeper.auth.sessions.login_timeout_s |
600 |
time allowed to finish a sign-in |
keeper.auth.sessions.idle_timeout_s |
3600 |
a session unused this long ends |
keeper.auth.sessions.absolute_timeout_s |
43200 |
a session ends this long after sign-in |
keeper.auth.sessions.touch_interval_s |
60 |
last activity written at most this often |
keeper.auth.sessions.refresh_tokens |
true |
refresh at the provider when the access token expires; a refusal ends the session, new group claims apply |
keeper.auth.sessions.refresh_default_interval_s |
300 |
when the token response has no expires_in |
keeper.auth.sessions.refresh_min_interval_s |
60 |
|
keeper.auth.sessions.refresh_lease_s |
30 |
one refresh per session at a time; retry delay after errors |
keeper.auth.sessions.metadata_cache_ttl_s |
3600 |
|
keeper.auth.sessions.csrf_header |
X-CSRF-Token |
CSRF token header on POST, PUT, PATCH, DELETE |
keeper.auth.sessions.allowed_origins |
[] |
Origin values accepted besides app_url's and redirect_uri's |
keeper.auth.sessions.purge_batch |
100 |
expired sessions deleted at each sign-in |
keeper.openapi
| Setting |
Default |
Description |
keeper.openapi.title |
Lorebase Keeper |
|
keeper.openapi.docs_enabled |
true |
serve <prefix>/docs and <prefix>/openapi.json |
keeper.domains
| Setting |
Default |
Description |
keeper.domains.leaf_size |
512 |
tree leaf size of new Domains, fixed for a Domain's life |
keeper.domains.creator_role |
admin |
role granted to the creator in a new Domain; null: none |
keeper.domains.protect_default_branch |
true |
the default branch accepts changes only through Proposals |
keeper.branches
| Setting |
Default |
Description |
keeper.branches.materialization |
full |
the only value: a copy holds full rows, copied from the source head or rebuilt from the tree of the start Revision; copy or overlay per branch: store.materialize.branch_mode |
keeper.branches.max_materialized_per_domain |
20 |
|
keeper.branches.name_pattern |
'^[A-Za-z0-9][A-Za-z0-9._-]*(/[A-Za-z0-9][A-Za-z0-9._-]*)*$' |
|
keeper.branches.max_name_length |
200 |
|
keeper.branches.lock_retry_after_s |
1 |
Retry-After of 503 branch-busy answers |
keeper.schema_changes
| Setting |
Default |
Description |
keeper.schema_changes.max_issues |
200 |
issues listed in one response |
keeper.schema_changes.max_listed_changes |
1000 |
element changes listed in one response |
keeper.schema_inference
POST /domains/{d}/schema:infer (docs/inference.md)
| Setting |
Default |
Description |
keeper.schema_inference.max_files |
50 |
files per request |
keeper.schema_inference.max_file_bytes |
1048576 |
per file (413 above); uploads also obey limits.max_request_body_bytes |
keeper.schema_inference.max_total_bytes |
1048576 |
all files together (413 above) |
keeper.schema_inference.file_suffixes |
[.csv, .tsv, .txt, .json, .jsonl, .ndjson, .parquet] |
|
keeper.schema_inference.file_formats |
{.json: json, .jsonl: jsonl, .ndjson: jsonl, .parquet: parquet} |
how a suffix is read: json (array of objects), jsonl (one object per line), parquet (needs lorebase-keeper[parquet]); other accepted suffixes are CSV/TSV text |
keeper.schema_inference.allowed_sources |
[] |
secret references of Postgres URLs callers may name, e.g. secret://env/ERP_DSN; empty disables Postgres inference; the default of every Domain not in domain_sources |
keeper.schema_inference.domain_sources |
{} |
per-Domain lists replacing allowed_sources for that Domain, e.g. {supply: [secret://env/ERP_DSN], hr: []}; [] = none |
keeper.schema_inference.allowed_schemas |
{} |
MySQL / SQL Server sources: schemas a source may be profiled in, e.g. {"secret://env/ERP_DSN": [dbo]}; unlisted = any |
keeper.schema_inference.preview_rows |
20 |
rows of a schema:preview when the request names none |
keeper.schema_inference.preview_max_rows |
200 |
most rows a preview may ask for |
keeper.schema_inference.preview_max_cell_chars |
1000 |
longest text returned per value of a preview |
keeper.schema_inference.max_issues |
500 |
issues listed in one response |
keeper.schema_inference.options
Thresholds and naming rules; a request may override them
| Setting |
Default |
Description |
keeper.schema_inference.options.sample_size |
10000 |
values read per column |
keeper.schema_inference.options.null_tokens |
["", "NULL", "null"] |
text read as null (after trimming) |
keeper.schema_inference.options.trim_text |
true |
|
keeper.schema_inference.options.max_evidence_samples |
5 |
sample values kept per evidence entry |
keeper.schema_inference.options.min_support_values |
20 |
non-null values for full confidence; fewer scale it down |
keeper.schema_inference.options.sampled_uniqueness_factor |
0.8 |
confidence factor when only a sample was read |
keeper.schema_inference.options.true_tokens |
["true", "t", "yes", "y"] |
boolean tokens, case-insensitive |
keeper.schema_inference.options.false_tokens |
["false", "f", "no", "n"] |
|
keeper.schema_inference.options.type_tolerance |
0.0 |
share of values allowed not to parse as the detected type |
keeper.schema_inference.options.mixed_type_report_ratio |
0.5 |
report text columns that mostly parse as one type |
keeper.schema_inference.options.leading_zero_integers_as_text |
true |
007 stays a text code |
keeper.schema_inference.options.decimal_max_scale |
4 |
more fractional digits: float |
keeper.schema_inference.options.decimal_precision_headroom |
2 |
integer digits added to the largest seen |
keeper.schema_inference.options.decimal_max_precision |
38 |
|
keeper.schema_inference.options.date_formats |
["%Y-%m-%d"] |
|
keeper.schema_inference.options.timestamp_formats |
["%Y-%m-%d %H:%M:%S", "%Y-%m-%dT%H:%M:%S"] |
without offset: UTC; RFC 3339 with offset is always recognized |
keeper.schema_inference.options.enum_max_cardinality |
12 |
enum: at most this many distinct values, |
keeper.schema_inference.options.enum_max_distinct_ratio |
0.5 |
at most this distinct/non-null ratio, |
keeper.schema_inference.options.enum_min_values |
10 |
at least this many non-null values, |
keeper.schema_inference.options.enum_max_value_length |
40 |
and short values (numbered codes such as a-1, a-2 never) |
keeper.schema_inference.options.text_min_length |
256 |
longer values: text |
keeper.schema_inference.options.text_name_hints |
[description, "*_description", summary, reason, notes, note, comment, comments, body, content, details] |
prose under these names: text |
keeper.schema_inference.options.text_hint_min_words |
3.0 |
|
keeper.schema_inference.options.string_length_buckets |
[16, 32, 64, 128, 256] |
max_length rounded up to a bucket |
keeper.schema_inference.options.propose_max_length |
true |
|
keeper.schema_inference.options.propose_patterns |
true |
e.g. ^S-[0-9]{4,}$ for code-like values |
keeper.schema_inference.options.pattern_min_values |
5 |
|
keeper.schema_inference.options.propose_non_negative_min |
true |
min 0 on numbers that are never negative |
keeper.schema_inference.options.required_max_null_ratio |
0.0 |
at most this share of nulls: required |
keeper.schema_inference.options.unique_min_values |
2 |
|
keeper.schema_inference.options.unique_name_patterns |
["*_number", "*_code", "*_id", "*_key", email, "*_email", "vat*", iban] |
unique non-key columns flagged unique only when named so |
keeper.schema_inference.options.key_name_patterns |
[id, key, code, "{table}_id", "{table}_key", "{table}_code", "{table}_number", "{table}_no", "*_id", "*_key", "*_code", "*_number", "*_no"] |
ranked primary key names |
keeper.schema_inference.options.allow_unnamed_primary_key |
true |
else the first unique, never-null column |
keeper.schema_inference.options.composite_key_max_columns |
2 |
|
keeper.schema_inference.options.minted_key_columns |
[key] |
key column of generated ids (ULID, UUID): minted keys |
keeper.schema_inference.options.min_inclusion_ratio |
0.95 |
share of a column's values found in a key for a Relation |
keeper.schema_inference.options.foreign_key_suffixes |
[_id, _key, _code, _number, _no, _ref, _fk] |
|
keeper.schema_inference.options.integer_reference_needs_name |
true |
|
keeper.schema_inference.options.weak_name_factor |
0.9 |
confidence factor: reference suffix without the target name |
keeper.schema_inference.options.no_name_factor |
0.75 |
confidence factor: no name hint at all |
keeper.schema_inference.options.detect_junction_tables |
true |
two references forming the key: a Relation |
keeper.schema_inference.options.junction_max_attributes |
8 |
|
keeper.schema_inference.options.one_to_one_needs_declared_unique |
true |
|
keeper.schema_inference.options.singular_kind_names |
true |
suppliers -> Supplier |
keeper.schema_inference.options.strip_table_prefixes |
[] |
e.g. [tbl_] |
keeper.schema_inference.options.title_names |
[name, title, label, display_name, "*_name", "*_title", "*_label"] |
|
keeper.schema_inference.options.propose_search |
true |
title searchable (A), text Attributes searchable (B) |
keeper.schema_inference.options.reserved_attribute_names |
[key, kind, ref, revision] |
renamed <kind>_<column> |
keeper.schema_inference.options.element_status |
draft |
status of proposed Kinds and Relations |
keeper.schema_inference.options.id_seed |
lorebase.infer |
same input and seed: same ids |
keeper.schema_inference.options.id_timestamp_ms |
0 |
|
keeper.schema_suggestions
GET /domains/{d}/schema/suggestions and :decide; thresholds in schema.suggestions (docs/extraction.md)
| Setting |
Default |
Description |
keeper.schema_suggestions.branch_prefix |
schema-suggestions/ |
accepted suggestions are applied on <prefix><id> |
keeper.schema_suggestions.max_decisions |
100 |
suggestions one request may decide |
keeper.operations
| Setting |
Default |
Description |
keeper.operations.max_bulk_rows |
10000 |
rows accepted by one import_records or import_links call |
keeper.operations.import_batch_rows |
1000 |
rows committed per Revision by bulk imports |
keeper.operations.max_bulk_delete_keys |
1000 |
keys accepted by one delete_records call (one Revision) |
keeper.operations.dry_run_max_edits |
1000 |
edits listed in a response; longer lists are cut |
keeper.operations.max_issues |
200 |
violations listed in one response |
keeper.operations.side_effect_max_edits |
100 |
edits listed in an Operation side effect event; longer lists are cut |
keeper.operations.direct_commit_on_protected |
false |
true: Operations with review policy never commit directly to protected branches |
keeper.operations.skip_expression_preconditions |
false |
with expressions off: true skips CEL preconditions (reported) instead of refusing |
keeper.operations.expressions_enabled |
true |
stored Operations evaluate CEL preconditions, review policies and steps (lorebase_core.expr: sandboxed, no I/O, deterministic, bounded below); false: refused. Whether schemas may hold CEL is schema.expressions_enabled |
keeper.operations.expression_limits
Budget of one Operation expression (ExprLimits)
| Setting |
Default |
Description |
keeper.operations.expression_limits.max_source_length |
4096 |
characters of expression source |
keeper.operations.expression_limits.max_depth |
48 |
nesting depth of the expression tree |
keeper.operations.expression_limits.max_steps |
10000 |
evaluation steps, comprehension iterations included |
keeper.operations.expression_limits.max_duration_ms |
250 |
wall-clock safety net behind max_steps |
keeper.operations.expression_limits.max_list_size |
10000 |
elements of a list or map built during evaluation |
keeper.operations.expression_limits.max_string_length |
65536 |
characters of a string built during evaluation |
keeper.operations.expression_limits.max_record_lookups |
100 |
Records and link lists read through record(), links() |
keeper.operations.expression_limits.decimal_precision |
34 |
significant digits of decimal arithmetic |
keeper.proposals
A Domain overrides required_approvals, required_approvals_from_stewards and required_roles with its own settings of those names; its merge_policy setting holds the merge policy options of new Proposals
| Setting |
Default |
Description |
keeper.proposals.required_approvals |
1 |
approvals needed to merge into a protected branch |
keeper.proposals.required_approvals_from_stewards |
0 |
of those, approvals from the Proposal's assignees (stewards, design 10.7); the larger of the two is needed |
keeper.proposals.required_roles |
[] |
roles that must each have approved (one approval from a holder who may review the Proposal); also a Domain setting |
keeper.proposals.allow_self_approval |
false |
authors may approve their own Proposals into protected branches |
keeper.proposals.changes_requested_blocks |
true |
an outstanding request_changes review blocks such merges |
keeper.proposals.auto_close_source_branch |
false |
delete the source branch after the merge (never the default or a protected branch) |
keeper.proposals.attempt_retention |
5 |
merge attempts kept per Proposal (merged ones always kept) |
keeper.proposals.preview_on_create |
true |
compute the merge preview (Conflict count) on creation |
keeper.proposals.max_resolutions |
1000 |
resolutions accepted by one bulk request |
keeper.proposals.max_item_decisions |
5000 |
extraction items one items:decide request may decide |
keeper.agent_writes
Operations that need review, written by agents (docs/agents.md) A request that names a task runs on the branch {branch_prefix}/{principal}/{task} and opens or updates a Proposal instead of answering 409 review-required. A Domain overrides any member with its own agent_writes setting.
| Setting |
Default |
Description |
keeper.agent_writes.enabled |
true |
false: review-required stays a plain 409 |
keeper.agent_writes.principal_types |
[service] |
principal types that are agents (user, service, api_key); the others must send auto_propose: true |
keeper.agent_writes.auto_propose |
true |
default of agents when a request leaves auto_propose out |
keeper.agent_writes.auto_merge |
false |
default when a request leaves auto_merge out |
keeper.agent_writes.allowed_roles_for_auto_merge |
[] |
roles whose agents may be merged automatically (the role is granted in the Domain and covers the branch); empty: nobody. Required approvals are never skipped |
keeper.agent_writes.max_open_branches_per_principal |
10 |
task branches one principal may hold at a time |
keeper.agent_writes.branch_prefix |
agent |
first part of task branch names |
keeper.agent_writes.branch_mode |
overlay |
task branches: overlay (only the rows the task changes, reads the target as of the fork) or copy (full rows) |
keeper.agent_writes.delete_branch_on_merge |
true |
delete the task branch when its Proposal merged or closed (not while another open Proposal uses it) |
keeper.agent_writes.max_task_length |
60 |
longest task slug (after normalization) |
keeper.agent_writes.title_template |
"Agent {principal}: {task}" |
title of the Proposal |
keeper.agent_writes.summary_parameter_values |
true |
list parameter values in the Proposal description; false: names only (values may be sensitive) |
keeper.agent_writes.summary_value_chars |
80 |
characters of a parameter value kept in the description |
keeper.agent_writes.max_summary_operations |
20 |
executions listed in one Proposal description |
keeper.reconciliation
Reconciliation queue endpoints (design 10.6); detection is set in scribe.reconciliation
| Setting |
Default |
Description |
keeper.reconciliation.default_limit |
50 |
candidates listed per page |
keeper.reconciliation.max_limit |
500 |
largest page a request may ask for |
keeper.reconciliation.bulk_max |
1000 |
candidates one bulk resolution may decide |
keeper.reconciliation.merge_check |
warn |
Proposal check of open candidates: off, warn (listed) or block (open contradictions stop the merge); the stricter of this and the Domain reconciliation_check applies (off < warn < block) |
keeper.documents
Document intake for extraction (docs/extraction.md)
| Setting |
Default |
Description |
keeper.documents.max_bytes |
20971520 |
largest document (20 MiB); the upload route allows this much |
keeper.documents.multipart_overhead_bytes |
65536 |
plus this, whatever limits.max_request_body_bytes says |
Accepted file name suffixes and their stored media type
| Setting |
Default |
Description |
keeper.documents.media_types..md |
text/markdown |
|
keeper.documents.media_types..markdown |
text/markdown |
|
keeper.documents.media_types..txt |
text/plain |
|
keeper.documents.media_types..pdf |
application/pdf |
|
keeper.documents.media_types..docx |
application/vnd.openxmlformats-officedocument.wordprocessingml.document |
|
keeper.documents (continued)
| Setting |
Default |
Description |
keeper.documents.url_hosts |
[] |
hosts documents may be fetched from (exact); empty: no URLs |
keeper.documents.url_schemes |
[https] |
|
keeper.documents.url_allow_private_targets |
false |
true: skip the global-address check and the pinning |
keeper.documents.url_timeout_s |
30.0 |
time budget of one fetch (redirects are not followed) |
keeper.documents.extract_on_upload |
true |
queue an extraction job per upload unless the request says no |
keeper.documents.jobs_listed |
20 |
extraction jobs shown with a document |
keeper.documents.max_text_chars |
2000000 |
parsed text served by GET .../documents/{id}/text (cut above) |
keeper.documents.max_source_key_chars |
1024 |
longest source_key of an upload (new bytes: a new version) |
keeper.stewards
Who looks after which part of a Domain (design 10.7); the roles come from the schema, assignees from grants
| Setting |
Default |
Description |
keeper.stewards.max_assignees_listed |
50 |
assignees shown per Proposal or candidate (principal.read) |
keeper.stewards.max_grants_scanned |
10000 |
grants of a Domain read to find assignees |
keeper.trace
Agent trace (docs/agent-trace.md): steps of agent sessions (query, counts, refs, latency; never values), written off the request path to agent_trace_steps. Needs the database section
| Setting |
Default |
Description |
keeper.trace.enabled |
false |
|
keeper.trace.session_header |
X-Lorebase-Session |
names the session; without it a traced request joins an inferred session (malformed: not traced) |
keeper.trace.tool_header |
X-Lorebase-Tool |
agent tool (MCP tool name) that made the request |
keeper.trace.opt_in_header |
X-Lorebase-Trace |
per request opt-in: on, or debug for query plans (callers holding trace.explain) |
keeper.trace.allow_header_opt_in |
true |
|
keeper.trace.trace_all |
false |
trace every authenticated request, except the web app's own |
keeper.trace.client_header |
X-Lorebase-Client |
names the client of a request; the web app sends web_client |
keeper.trace.web_client |
web |
client_header value of the web app's requests |
keeper.trace.include_web |
false |
true: trace_all also traces the web app (requests marked with client_header, and browser session cookie requests that name no session) |
keeper.trace.principals |
[] |
principal ids whose requests are traced |
keeper.trace.api_keys |
[] |
API key ids whose requests are traced |
keeper.trace.inferred_session_idle_s |
300.0 |
requests without a session id are grouped per principal and client (API key or web session, user agent) while they come within this gap (either side); marked inferred |
keeper.trace.capture_filter_values |
non_sensitive |
filter values kept: non_sensitive (on fields not sensitive under the effective tags, through Traits and Derivations; key and title by their Attributes) or none |
keeper.trace.max_filter_value_items |
20 |
items kept of a list compared by a filter |
keeper.trace.capture_literals |
false |
true: keep every filter value, sensitive ones included |
keeper.trace.capture_search_text |
true |
keep search text (q, search, match); false: its length only |
keeper.trace.max_query_bytes |
16384 |
larger request bodies are recorded as truncated |
keeper.trace.max_text_chars |
500 |
longest string kept in a recorded query |
keeper.trace.max_response_bytes |
1048576 |
larger responses are recorded without counts and refs |
keeper.trace.max_refs |
500 |
Record and Link refs kept per step |
keeper.trace.max_note_chars |
4000 |
longest note an agent attaches |
keeper.trace.max_steps_per_session |
2000 |
per session and principal; later steps are not written |
keeper.trace.retention_days |
7.0 |
steps older than this are deleted |
keeper.trace.prune_interval_s |
300.0 |
|
keeper.trace.prune_batch |
10000 |
steps deleted per pass at most |
keeper.trace.queue_size |
10000 |
past it steps are dropped (lorebase.trace.steps.dropped) |
keeper.trace.batch_size |
200 |
steps per transaction |
keeper.trace.flush_interval_s |
1.0 |
longest wait before queued steps are written |
keeper.trace.write_timeout_s |
5.0 |
a failed batch is dropped and counted |
inference
Model inference, used by Scribe only. Lorebase never imports a model vendor library: every chat model is an OpenAI-compatible HTTP endpoint, local or hosted.
inference.llm
Tier 2; env aliases LLM_BASE_URL, LLM_MODEL, LLM_API_KEY
| Setting |
Default |
Description |
inference.llm.provider |
openai_compatible |
openai_compatible | fake |
inference.llm.base_url |
http://localhost:11434/v1 |
local Ollama; in docker compose: http://ollama:11434/v1 |
inference.llm.model |
qwen2.5:7b-instruct |
any model name the endpoint serves |
inference.llm.api_key |
null |
hosted endpoints: secret://env/LLM_API_KEY |
inference.llm.structured_output |
json_schema |
json_schema | json_mode. json_schema sends the schema as response_format and steps down (json_mode, then prompt only) when the server rejects it; answers are always validated |
inference.llm.json_schema_strict |
null |
response_format.json_schema.strict; null leaves it out |
inference.llm.repair_attempts |
1 |
repair calls after an invalid answer (0-3) |
inference.llm.temperature |
0 |
null leaves it out, for models that refuse the parameter |
inference.llm.max_tokens |
null |
completion cap; null leaves it to the server |
inference.llm.max_concurrency |
2 |
|
inference.llm.timeout_s |
120 |
|
inference.llm.trust_env |
false |
false: proxy variables ignored; true: HTTP(S)_PROXY, NO_PROXY honored |
inference.llm.retry
429, 5xx and connection errors; Retry-After is honored
| Setting |
Default |
Description |
inference.llm.retry.max_retries |
3 |
|
inference.llm.retry.backoff_base_s |
0.5 |
wait before retry n: min(backoff_max_s, base * 2**n), jittered |
inference.llm.retry.backoff_max_s |
30 |
|
inference.llm (continued)
| Setting |
Default |
Description |
inference.llm.price_per_million_tokens |
{input: 0, output: 0} |
zero for local models |
inference.llm.headers |
{} |
extra HTTP headers; values accept secret references |
inference.llm.extra_body |
{} |
extra request fields, e.g. {reasoning_effort: none} |
inference.llm.tasks |
{} |
per-task overrides, see the commented example below |
Example:
# Hosted OpenAI-compatible endpoint (OpenAI, Azure OpenAI, Mistral, ...):
# base_url: https://api.openai.com/v1
# model: <model name>
# api_key: secret://env/LLM_API_KEY
# price_per_million_tokens: {input: <price>, output: <price>}
#
# Anthropic through its OpenAI-compatible endpoint (docs/inference-providers.md):
# base_url: https://api.anthropic.com/v1/
# model: claude-haiku-5-5
# api_key: secret://env/ANTHROPIC_API_KEY
# headers: {anthropic-workspace-id: secret://env/ANTHROPIC_WORKSPACE_ID} # org-level keys
# temperature: null # refused by current models
# json_schema_strict: true # required by the endpoint
# extra_body: {thinking: {type: disabled}}
# max_tokens: 4096
#
# LiteLLM proxy (compose profile `litellm`), reaching Anthropic, Google, Bedrock, ...:
# base_url: http://litellm:4000/v1
# model: <model alias defined in the proxy>
# api_key: secret://env/LITELLM_KEY
#
# Small local model for bulk extraction, larger hosted model for schema discovery:
# tasks:
# extraction: {model: qwen2.5:7b-instruct}
# schema_discovery:
# base_url: https://api.openai.com/v1
# model: <model name>
# api_key: secret://env/HOSTED_KEY
inference.embeddings
| Setting |
Default |
Description |
inference.embeddings.provider |
fastembed |
fastembed | openai_compatible | fake |
inference.embeddings.model |
BAAI/bge-small-en-v1.5 |
|
inference.embeddings.dims |
384 |
must match the model; the vector index uses it |
inference.embeddings.batch_size |
32 |
texts per model call or request |
inference.embeddings.query_prefix |
"" |
put before a search query (lorebase-inference embed --query); E5: "query: " |
inference.embeddings.document_prefix |
"" |
put before every Record text Scribe embeds; E5: "passage: " |
inference.embeddings.cache_dir |
null |
fastembed: where model files are stored (library default) |
inference.embeddings.threads |
null |
fastembed: CPU threads (library default) |
inference.embeddings.base_url |
null |
required for openai_compatible, e.g. http://localhost:11434/v1 |
inference.embeddings.api_key |
null |
|
inference.embeddings.timeout_s |
60 |
openai_compatible request timeout |
inference.embeddings.trust_env |
false |
false: proxy variables ignored; true: HTTP(S)_PROXY, NO_PROXY honored |
inference.embeddings.retry |
{max_retries: 3, backoff_base_s: 0.5, backoff_max_s: 30} |
|
Example:
# Ollama embeddings instead of fastembed:
# provider: openai_compatible
# base_url: http://localhost:11434/v1
# model: nomic-embed-text
# dims: 768
Tier 1, in-process on CPU; optional extra [local-extractors]
| Setting |
Default |
Description |
inference.local_extractors.provider |
gliner |
gliner | fake |
inference.local_extractors.relation_provider |
glirel |
glirel | cooccurrence | fake (fake when provider is fake) |
inference.local_extractors.gliner |
{enabled: true, model: urchade/gliner_multi-v2.1, threshold: 0.5, cache_dir: null, window_chars: 1500} |
Check the license of each checkpoint's weights before enabling it: the libraries are Apache-2.0, the weights are separate artifacts with their own terms. |
inference.local_extractors.glirel |
{enabled: true, model: jackboyla/glirel-large-v0, threshold: 0.5, cache_dir: null, window_chars: 1500} |
|
inference.local_extractors.cooccurrence |
{max_pairs_per_sentence: 200} |
relation_provider: cooccurrence (no model) |
inference.local_extractors.spacy |
{enabled: false, pipeline: en_core_web_sm} |
|
inference (continued)
| Setting |
Default |
Description |
inference.mode_memory_key |
null |
secret keying the digests of remembered structured output modes, e.g. secret://env/LOREBASE_MODE_KEY (changing it forgets) |
inference.mode_memory_ttl_s |
604800 |
seconds a remembered mode counts (7 days) |
inference.fake
Deterministic provider for tests and offline runs: set provider: fake in llm, embeddings and local_extractors. No network access is needed.
| Setting |
Default |
Description |
inference.fake.fixtures_dir |
null |
directory of <prompt-hash>.json canned chat answers |
inference.fake.dictionary |
{} |
label -> surface forms matched by the fake span extractor |
inference.fake.max_schema_depth |
6 |
nesting depth beyond which optional properties are omitted |
database
Postgres 16 with the pgvector extension. Environment: LOREBASE__DATABASE__DSN, ...
| Setting |
Default |
Description |
database.dsn |
postgresql+asyncpg://lorebase@localhost:5432/lorebase |
alias LOREBASE_DB_DSN; e.g. secret://env/LOREBASE_DB_DSN. postgresql:// URLs are switched to the asyncpg driver; without a password, PGPASSWORD or ~/.pgpass is used |
database.pool_size |
10 |
connections kept open per process |
database.max_overflow |
10 |
extra connections under load |
database.pool_timeout_s |
30 |
wait for a free pooled connection |
database.pool_recycle_s |
1800 |
replace older connections; -1 never |
database.statement_timeout_ms |
30000 |
per statement; 0 disables |
database.migration_statement_timeout_ms |
0 |
while migrations run; 0 disables |
database.application_name |
lorebase |
shown in pg_stat_activity |
database.maintenance_dsn |
null |
optional secret: a role that owns mat_records, used by the job worker for per-Attribute index DDL queued by roles that do not own it. Alias LOREBASE_DB_MAINTENANCE_DSN. null: such roles skip index management (python -m lorebase_store.indexes plan) |
database.application_role |
null |
role the services connect as; revisions 0005, 0008 and 0010 grant it EXECUTE on the security definer read functions (store schema notes); 0008 and 0010 refuse to run while it may CREATE in the schema of the tables or of pg_trgm. null grants nothing; the role must exist before migrating |
database.read_dsn |
[] |
read replicas (docs/deployment.md): one URL, a comma-separated string or a list; literals or secret references, e.g. [secret://env/LOREBASE_DB_READ_DSN]. Alias LOREBASE_DB_READ_DSN. Reads are spread round-robin; empty keeps every read on the primary. Each replica gets its own pool, sized as above |
database.read_max_lag_ms |
5000 |
a replica lagging more than this serves no reads; 0: unchecked |
database.read_fallback_to_primary |
true |
reads no replica can serve go to the primary; false: 503 |
database.read_probe_timeout_ms |
2000 |
budget to reach a replica and run its freshness probe |
database.prepared_statement_cache_size |
500 |
prepared statements per connection (asyncpg); 0 disables |
store
Storage engine. layout and embeddings are read once by the initial migration; the other keys are read at runtime.
store.layout
| Setting |
Default |
Description |
store.layout.hash_partitions |
8 |
domain_id hash partitions of objects, tree_nodes, mat_records and mat_links |
store.storage
Storage profile (docs/performance.md#storage-profiles): memory and disk traded for speed. Applied by python -m lorebase_store.storage apply (batched, resumable, reads go on); until then the database keeps what it has. Each key but profile overrides one choice.
| Setting |
Default |
Description |
store.storage.profile |
balanced |
compact | balanced | fast |
store.storage.lineage_maps |
null |
shared | inline (null: inline with fast, shared otherwise) |
store.storage.values_index |
null |
GIN index on Record values (null: off with compact) |
store.storage.attribute_indexes |
null |
all | constraints (null: constraints with compact) |
store.storage.lexicon_index |
null |
trigram index of the typo lexicon (null: off with compact) |
store.storage.embedding_precision |
null |
half | full HNSW index (null: half with compact) |
store.storage.hnsw_m |
null |
HNSW links per node, 2-100 (null: pgvector's 16) |
store.storage.hnsw_ef_construction |
128 |
HNSW build candidate list, >= 2 * hnsw_m (null: pgvector's 64); both apply to indexes built from then on; an existing index keeps its own until rebuilt (docs/operations.md) |
store.storage.compression |
null |
lz4 | pglz for values above ~2 kB (null: pglz with compact) |
store.storage.records_fillfactor |
null |
50-100 (null: 90 with fast, 100 otherwise) |
store.storage.prewarm |
null |
apply ends with storage prewarm (null: on with fast) |
store.storage.bm25_index |
null |
BM25 index of search; needs pg_search (null: off in every profile) |
store.storage.prewarm_fraction |
0.8 |
share of shared_buffers the prewarm may fill |
store.storage.batch_rows |
2000 |
rows rewritten per transaction by storage apply |
store.storage.lock_timeout_ms |
2000 |
wait for a table lock in storage apply, then retry later |
store.embeddings
| Setting |
Default |
Description |
store.embeddings.dims |
384 |
HNSW index dimension; must equal inference.embeddings.dims |
store.values
Value index of POST /resolve (design 8.9): distinct short values per branch, kept by triggers on mat_records. Enums and resolvable: true Attributes are always held; sensitive, vault, resolvable: false and non-string Attributes never; plain strings while their distinct values on the default branch stay within the thresholds below.
| Setting |
Default |
Description |
store.values.enabled |
true |
false drops every selection on the next pass |
store.values.max_value_chars |
64 |
longer values are left out |
store.values.max_distinct |
1000 |
a plain string with more distinct values is not held |
store.values.max_distinct_ratio |
0.2 |
...nor one whose distinct values exceed this share of Records |
store.values.ratio_min_records |
50 |
Records a Kind needs before max_distinct_ratio applies |
store.values.hysteresis |
1.2 |
a held plain string is dropped only past this factor |
store.values.hard_max_distinct |
10000 |
values of one Attribute any branch may hold, enums included |
store.values.stats_interval_s |
3600.0 |
distinct values counted again this often (schema change: at once) |
store.values.max_added_per_pass |
20 |
Attributes one pass adds; the rest go to the next pass |
store.values.pass_timeout_ms |
120000 |
budget of a pass once it holds the Domain's lock (writers wait) |
store.resolve_misses
Phrases POST /resolve matched to nothing (design 8.10), kept for vocabulary suggestions. They are free text typed by callers: off by default, pruned after retention_days, deleted with their Domain (lorebase-scribe misses purge DOMAIN deletes them at once).
| Setting |
Default |
Description |
store.resolve_misses.enabled |
false |
Keeper records, Scribe prunes and consolidates |
store.resolve_misses.retention_days |
30.0 |
a phrase not seen for this long is deleted |
store.resolve_misses.min_phrase_chars |
3 |
shorter phrases are not recorded |
store.resolve_misses.max_phrase_chars |
64 |
longer ones neither (an alias is at most 128 characters) |
store.resolve_misses.ignore_words |
[] |
filler words never recorded alone, e.g. [how, many, the] |
store.resolve_misses.max_per_domain |
10000 |
phrases kept per Domain; a new one evicts the least missed (no more missed than it) or is dropped |
store.resolve_misses.max_phrases_per_caller |
500 |
phrases one caller alone may hold per Domain |
store.resolve_misses.max_callers |
20 |
distinct callers counted per phrase (digests, never names) |
store.resolve_misses.caller_key |
null |
REQUIRED when enabled: >= 32 characters, e.g. secret://env/LOREBASE_MISS_CALLER_KEY; keys the HMAC-SHA256 caller digests. Changing it counts distinct callers afresh |
store.resolve_misses.prune_batch |
5000 |
expired phrases deleted per statement |
store.coverage
Records and missing values per Kind of the default branch head, behind the coverage block of queries and resolutions (design 8.11); counted by Scribe, looked up by reads.
| Setting |
Default |
Description |
store.coverage.enabled |
true |
|
store.coverage.max_records_per_kind |
1000000 |
larger Kinds are counted over a sample (marked approximate) |
store.coverage.statement_timeout_ms |
120000 |
per Kind; a Kind that times out keeps its statistics |
store.coverage.min_interval_s |
300.0 |
a Kind counted more recently is not recounted (unseen Records are) |
store.writes
| Setting |
Default |
Description |
store.writes.insert_batch_rows |
1000 |
rows per multi-row INSERT |
store.writes.copy_threshold_rows |
5000 |
larger batches use COPY into a staging table; 0 disables COPY |
store.nodes
| Setting |
Default |
Description |
store.nodes.cache_bytes |
67108864 |
in-process LRU cache of tree nodes (64 MiB); 0 disables it |
store.nodes.prefetch_batch |
1000 |
node hashes per prefetch query |
store.commit
| Setting |
Default |
Description |
store.commit.lock_timeout_ms |
10000 |
wait for the branch lock before failing; 0 waits forever |
store.commit.prefetch_batch |
1000 |
keys per query when validation prefetches materialized rows |
store.materialize
| Setting |
Default |
Description |
store.materialize.text_search_config |
simple |
Postgres text search configuration of Record search vectors |
store.materialize.read_batch_rows |
5000 |
rows or object bodies per query in rebuild and verification |
store.materialize.verify_max_divergences |
1000 |
divergences listed by a verification report |
store.materialize.derivations_enabled |
true |
recompute expression and aggregate Derivations on every commit, merge and rebuild (derived column, not the tree) |
store.materialize.fast_forward_history_max_revisions |
1000 |
Revisions of a fast-forward written to the history one by one; longer: one row at the source head |
store.materialize.derivation_max_incremental_records |
10000 |
Records one Derivation recomputes per commit; above, every Record of its Kind |
store.materialize.branch_mode |
copy |
default mode of new branches: copy (full rows) or overlay (only changed rows over the base, docs/store-schema-notes.md) |
store.materialize.overlay_branch_prefixes |
[agent/, extract/, source/] |
these default to overlay |
store.materialize.overlay_max_rows |
50000 |
an overlay holding more rows is promoted to a full copy |
store.materialize.overlay_max_base_lag_revisions |
1000 |
generations the base may move past the fork point before the overlay is promoted |
store.materialize.overlay_max_per_branch |
32 |
open overlays per base branch; further branches are copies |
store.materialize.async_copy_min_rows |
10000 |
larger copies run as jobs: from an overlay, or built from the tree (overlay sources, older Revisions) |
store.materialize.async_delete_min_rows |
10000 |
larger branches are hidden at once, a job deletes the rows |
store.materialize.job_batch_rows |
5000 |
most rows per transaction of a branch copy or deletion job |
store.materialize.job_batch_min_rows |
100 |
fewest rows per batch; batches start at this size |
store.materialize.job_batch_target_ms |
200 |
branch lock hold a batch aims at while no commit waits |
store.materialize.job_batch_contended_ms |
50 |
hold a batch aims at once a commit waited for the lock |
store.materialize.job_concurrency |
2 |
branch copy and deletion jobs running at once (all workers) |
store.materialize.job_retry_seconds |
10 |
delay before a branch job that had to wait runs again |
store.materialize.copy_max_failures |
3 |
failed copy jobs before an overlay is rebuilt from its tree |
store.materialize.copy_retry_max_seconds |
3600 |
longest delay before a failed copy is retried |
store.gc
Garbage collection (docs/operations.md)
| Setting |
Default |
Description |
store.gc.grace_seconds |
3600 |
Revisions younger than this are kept with all they reach |
store.gc.mark_batch |
1000 |
tree node hashes per query while marking |
store.gc.sweep_batch_rows |
5000 |
rows deleted per transaction while sweeping |
store.gc.guard_branch_locks |
true |
hold every branch lock (try-lock) around each sweep batch |
store.gc.max_guarded_branches |
500 |
refs one sweep batch may lock; more refs need a larger value |
store.gc.guard_wait_seconds |
60 |
wait for in-flight commits before the collection stops |
store.gc.guard_retry_ms |
200 |
pause between lock attempts |
store.jobs
Job queue workers
| Setting |
Default |
Description |
store.jobs.max_attempts |
3 |
runs of a job before it is marked failed |
store.jobs.retry_delay_seconds |
60 |
delay before a failed job runs again |
store.jobs.stale_after_seconds |
90 |
running jobs without heartbeat for this long are reclaimed (worker gone) |
store.outbox
Outbox events, shared by every consumer (design 10.5)
| Setting |
Default |
Description |
store.outbox.retention_s |
604800.0 |
events every consumer handed off are kept this long (7 days) |
store.outbox.max_age_s |
2592000.0 |
events are deleted after this even when still pending (30 days) |
store.outbox.prune_batch |
10000 |
events deleted per reason by one batch (own transaction) |
store.outbox.prune_budget_s |
5.0 |
seconds one pruning pass keeps deleting batches |
store.bundle
Domain export and import (docs/operations.md)
| Setting |
Default |
Description |
store.bundle.batch_rows |
5000 |
rows read, written or verified per batch |
store.bundle.compression |
gzip |
gzip | none, applied to the .jsonl files |
store.bundle.compression_level |
6 |
gzip level 1-9 |
store.bundle.verify_after_import |
true |
verify rebuilt branches against their trees |
store.bundle.max_member_bytes |
17179869184 |
import: largest file, counted as read after gzip (16 GiB) |
store.bundle.max_total_bytes |
68719476736 |
import: largest sum of file sizes (64 GiB) |
store.bundle.max_entries |
100000 |
import: most files and directories in a bundle |
store.bundle.max_line_bytes |
67108864 |
import: longest .jsonl record and largest .json file (64 MiB) |
store.profiling
Profilers feeding schema inference (docs/inference.md)
store.profiling.csv
| Setting |
Default |
Description |
store.profiling.csv.sample_rows |
10000 |
rows kept per file; every row is still counted |
store.profiling.csv.sampling |
reservoir |
head (first rows) | reservoir (uniform, seeded, file order) |
store.profiling.csv.seed |
0 |
reservoir seed: the same file gives the same rows |
store.profiling.csv.encodings |
[utf-8-sig, cp1252] |
tried in order |
store.profiling.csv.sniff_bytes |
65536 |
leading bytes used to detect the delimiter |
store.profiling.csv.delimiters |
",;\t\|" |
delimiters the sniffer may pick; .tsv files use a tab |
store.profiling.csv.max_columns |
500 |
wider files are refused |
store.profiling.records
Rows read as records: JSON, JSON Lines, Parquet, HTTP JSON
| Setting |
Default |
Description |
store.profiling.records.sample_rows |
10000 |
rows kept per table; every row is still counted |
store.profiling.records.sampling |
reservoir |
head (first rows) | reservoir (uniform, seeded, read order) |
store.profiling.records.seed |
0 |
|
store.profiling.records.max_columns |
500 |
tables with more distinct columns are refused |
store.profiling.records.batch_rows |
10000 |
rows decoded per batch of an uploaded Parquet file |
store.profiling.postgres
| Setting |
Default |
Description |
store.profiling.postgres.sample_rows |
10000 |
rows read per table (LIMIT), in a read-only transaction |
store.profiling.postgres.max_tables |
200 |
tables profiled per request |
store.profiling.postgres.statement_timeout_ms |
15000 |
|
store.profiling.postgres.exact_row_count |
false |
true: count(*) per table; false: planner estimate |
store.profiling.sql
MySQL, MariaDB and SQL Server tables (drivers: keeper extras)
| Setting |
Default |
Description |
store.profiling.sql.sample_rows |
10000 |
rows read per table (LIMIT / TOP) |
store.profiling.sql.max_tables |
50 |
tables profiled per request |
store.profiling.sql.total_timeout_s |
120 |
all the tables of a request |
store.profiling.sql.max_value_chars |
1000 |
longer text is cut; unbounded binary and unknown types are skipped |
store.profiling.sql.statement_timeout_ms |
15000 |
|
store.profiling.sql.connect_timeout_s |
10.0 |
|
store.profiling.sql.count_rows |
true |
count(*) per table so a partial sample is known as such |
store.indexes
Per-Attribute partial indexes on mat_records
| Setting |
Default |
Description |
store.indexes.enabled |
true |
false: an administrator manages them. A commit role that does not own mat_records queues them for the job worker when database.maintenance_dsn is set, else skips them (warning) |
store.indexes.inline_max_rows |
50000 |
larger Kinds get their indexes from a queued concurrent build |
store.indexes.job_batch |
10 |
queued index jobs claimed per runner pass |
store.reads
| Setting |
Default |
Description |
store.reads.schema_cache_entries |
256 |
compiled schemas per process, keyed by head Revision id |
store.reads.max_page_rows |
500 |
largest page of Records, Links or query results |
store.reads.max_search_rows |
100 |
largest full-text search result |
store.reads.search_candidates |
1000 |
matches ranked per search (bounds very common terms) |
store.reads.max_hydrate_keys |
1000 |
Records per batched hydration |
store.reads.use_definer_functions |
true |
search, incoming Links, neighborhoods and attribute queries through the security definer functions, whose index plans survive row-level security; false: the same SQL, plain |
store.reads.definer_ceilings
Largest counts the definer functions accept (NULL and values outside 1..ceiling are refused). Read when revisions 0008, 0010 run, later changes have no effect; each must be at least the cap above (traverse below) that the read primitives pass
| Setting |
Default |
Description |
store.reads.definer_ceilings.limit |
10000 |
rows of a page or search: >= max_page_rows, max_search_rows |
store.reads.definer_ceilings.candidates |
100000 |
search candidates: >= search_candidates |
store.reads.definer_ceilings.depth |
10 |
neighborhood hops: >= traverse.max_depth |
store.reads.definer_ceilings.fanout |
10000 |
Links per Record and hop: >= traverse.max_fanout |
store.reads.definer_ceilings.max_steps |
1000000 |
Links walked plus seeds: >= traverse.max_links + max_seeds |
store.reads.definer_ceilings.max_nodes |
100000 |
Records of a neighborhood: >= traverse.max_nodes |
store.traverse
Neighborhood caps and path Derivation fan-out
| Setting |
Default |
Description |
store.traverse.max_depth |
3 |
hops |
store.traverse.max_seeds |
50 |
seed Records |
store.traverse.max_fanout |
50 |
Links followed per Record and hop |
store.traverse.max_links |
2000 |
Links walked, breadth first |
store.traverse.max_nodes |
200 |
Records returned, nearest first |
store.traverse.path_fanout |
100 |
Links a path Derivation follows per intermediate Record |
schema
Static validation of schemas, shared by Keeper and the store: schema change sets, the commit pipeline, merges and schema inference apply the same rules.
| Setting |
Default |
Description |
schema.expressions_enabled |
true |
accept Operations and Derivations written in CEL (parsed and type checked); false refuses them (schema.expressions_disabled) |
schema.expression_limits
Budget of one schema expression: compiled at validation, evaluated by the materializer for Derivations
| Setting |
Default |
Description |
schema.expression_limits.max_source_length |
4096 |
|
schema.expression_limits.max_depth |
48 |
|
schema.expression_limits.max_steps |
10000 |
|
schema.expression_limits.max_duration_ms |
250 |
|
schema.expression_limits.max_list_size |
10000 |
|
schema.expression_limits.max_string_length |
65536 |
|
schema.expression_limits.max_record_lookups |
100 |
|
schema.expression_limits.decimal_precision |
34 |
|
schema.suggestions
Schema suggestions from extraction (docs/extraction.md)
| Setting |
Default |
Description |
schema.suggestions.min_support |
2 |
observations before a pending suggestion is listed |
schema.suggestions.max_suggestions |
50 |
listed at most, the best supported first |
schema.suggestions.max_examples |
3 |
evidence examples kept per suggestion |
schema.suggestions.max_samples |
20 |
distinct sample values kept per suggestion |
schema.suggestions.max_labels |
5 |
labels (as written) kept per suggestion |
schema.suggestions.max_tracked |
2000 |
extraction suggestions kept per Domain; new ones beyond are dropped |
schema.suggestions.max_vocabulary_tracked |
500 |
pending vocabulary suggestions (resolution misses), counted apart |
schema.suggestions.name_similarity |
0.85 |
least similarity of a name to an existing element's |
schema.suggestions.synonyms |
[[supplier, vendor], [contract, agreement], [part, component, item], [plant, site, facility], [purchase_order, order], [customer, client], [quantity, qty]] |
names meaning the same element (snake_case, singular) |
schema.suggestions.merge_similarity |
0.6 |
least word overlap (Jaccard) for two Attribute suggestions of one Kind with samples of one value type to count as one |
schema.suggestions.kind_stoplist |
[section, clause, article, paragraph, chapter, annex, appendix, schedule, exhibit, attachment, page, table, figure, document, proposal] |
|
schema.suggestions.kind_min_attributes |
2 |
generic words never suggested as a Kind Attribute suggestions (or observations) a new Kind needs before it is listed; 0 lists every Kind |
schema.suggestions.kind_independent_instances |
2 |
distinct codes that make a Kind an entity of its own, kept although an Attribute has its name; 0 turns it off |
schema.suggestions.kind_independent_attributes |
2 |
Attributes owned by a proposed Kind with the same effect |
Example:
# type_inference: options of schema inference (as keeper.schema_inference.options) that
# type new Attributes from their samples; defaults differ in min_support_values: 5,
# enum_min_values: 5 and propose_max_length: false
read
Read path of Keeper (design 8). Reads hit the materialized head of a branch; at=<revision> reads go through the Revision tree.
read.records
Pages of Records and Links
| Setting |
Default |
Description |
read.records.default_limit |
50 |
|
read.records.max_limit |
500 |
|
read.log
Pages of a branch log
| Setting |
Default |
Description |
read.log.default_limit |
50 |
|
read.log.max_limit |
500 |
|
read.history
Per-Record history pages and lineage
| Setting |
Default |
Description |
read.history.default_limit |
50 |
|
read.history.max_limit |
500 |
|
read.history.lineage_scan_changes |
1000 |
newest changes searched for rows without lineage revisions |
read.history.lineage_count_max |
10000 |
changes counted; beyond, changes_truncated is true |
read (continued)
| Setting |
Default |
Description |
read.statement_timeout_ms |
2000 |
statement_timeout of read transactions; 0 disables |
read.schema_cache_entries |
64 |
compiled schemas kept in memory, by schema hash |
read.query
Limits of the query DSL
| Setting |
Default |
Description |
read.query.max_via_depth |
3 |
nesting of via and count predicates |
read.query.max_predicates |
64 |
|
read.query.max_in_values |
1000 |
|
read.query.max_order_terms |
4 |
|
read.query.max_includes |
5 |
Relations in include |
read.query.include_default_limit |
10 |
neighbors per Record in include |
read.query.include_max_limit |
50 |
|
read.query.max_aggregate_groups |
1000 |
groups returned by aggregate; more are cut |
read.traverse
POST /traverse and POST /neighborhood; a neighborhood is also bounded by store.traverse (the smaller cap applies)
| Setting |
Default |
Description |
read.traverse.max_hops |
5 |
|
read.traverse.max_nodes_per_hop |
1000 |
|
read.traverse.max_seeds |
100 |
|
read.traverse.max_depth |
3 |
|
read.traverse.max_nodes |
2000 |
|
read.traverse.max_edges |
5000 |
|
read.time_travel
| Setting |
Default |
Description |
read.time_travel.max_scan_entries |
100000 |
largest Kind or Relation partition listed at a Revision |
read.time_travel.snapshot_max_entries |
100000 |
largest Revision (Records + Links) materialized for filters, search, traversal, history and health at it; larger refused |
read.time_travel.snapshot_max_history_revisions |
100000 |
deeper histories (generation + 1) are refused |
read.time_travel.snapshot_max_per_domain |
4 |
snapshots per Domain (being deleted included); oldest unused dropped first, 503 when none can go |
read.time_travel.snapshot_min_age_s |
60 |
grace period before a snapshot may be dropped |
read.time_travel.snapshot_max_concurrent_builds |
2 |
per process; each uses a second DB connection |
read.time_travel.snapshot_build_wait_ms |
5000 |
wait for a build slot before answering 503 |
read.time_travel.snapshot_builds_per_minute |
6 |
per principal and process (429 beyond) |
read.time_travel.snapshot_build_burst |
3 |
|
read.diff
| Setting |
Default |
Description |
read.diff.max_changes |
100000 |
larger diffs are refused; narrow them with kinds/relations |
read.diff.page
| Setting |
Default |
Description |
read.diff.page.default_limit |
100 |
|
read.diff.page.max_limit |
1000 |
|
read.health
GET .../health and .../kinds/{kind}/health (docs/agents.md)
| Setting |
Default |
Description |
read.health.max_records_validated |
5000 |
Records of a Kind re-validated; larger Kinds are sampled |
read.health.examples_per_issue |
3 |
Record keys listed per kind of violation |
read.health.max_issue_groups |
50 |
kinds of violation listed per Kind |
read.health.duplicates_max_records |
200000 |
larger Kinds skip the duplicate check; 0 turns it off |
read.health.max_duplicate_groups |
20 |
duplicate groups listed per Attribute |
read.health.duplicate_keys_per_group |
5 |
|
read.health.orphan_examples |
5 |
orphan Links and Records with a gap listed per Relation |
read.health.statement_timeout_ms |
15000 |
for the scans, instead of read.statement_timeout_ms; 0 disables |
read.coverage
The coverage block of POST /query and /resolve (design 8.11)
| Setting |
Default |
Description |
read.coverage.enabled |
true |
default of requests that do not say coverage |
read.coverage.note_min_share |
0.01 |
least share of Records missing an Attribute for a note |
read.coverage.max_unseen_age_s |
3600.0 |
older counts of Records unseen by Sources: approximate |
search
Search (design 8.4). Keeper never computes embeddings: semantic and hybrid search take the caller's query_vector. The text search configuration is store.materialize.text_search_config and the vector dimension store.embeddings.dims.
search.results
| Setting |
Default |
Description |
search.results.default_limit |
20 |
|
search.results.max_limit |
100 |
|
search (continued)
| Setting |
Default |
Description |
search.candidates |
200 |
rows of each ranked list before fusion and paging |
search.backend |
postgres |
Ranking of lexical search without filters: postgres (ts_rank_cd over the GIN index) or bm25 (the BM25 index of ParadeDB pg_search, an optional AGPL-3.0 extension; build it with store.storage.bm25_index, docs/search.md#bm25-ranking). Without it search keeps postgres. |
search.trigram_threshold |
0.3 |
Typo tolerance: a query word the branch does not hold is corrected to its closest words in the branch's word lexicon (trigram similarity) and searched as (word | correction ...). least similarity of a correction to the query word |
search.trigram_weight |
0.5 |
weight of ts_rank_cd for hits found only through corrections (they always rank after the hits of q as written) |
search.typo_candidates |
3 |
corrections tried per unknown word; 0 turns typo tolerance off |
search.typo_min_length |
4 |
shortest query word (characters) that is corrected |
search.common_term_docs |
2000 |
Lexical search without filters: the word lexicon bounds the matches of q. Within common_term_docs every match is ranked; when every term is more common, only the first common_term_candidates matches in index order are, and the response says truncated_candidates. 0 ranks store.reads.search_candidates matches whatever the terms. |
search.common_term_candidates |
1000 |
|
search.headline_options |
"MaxWords=30, MinWords=10, MaxFragments=2" |
ts_headline of snippets |
search.partial
Partial matching: when q as written (every term) finds fewer than min_results Records, Records holding some of its terms follow, after every full match (match: partial).
| Setting |
Default |
Description |
search.partial.min_results |
1 |
full matches below which partial ones are added; 0 turns it off |
search.partial.weight |
0.5 |
score: weight * (ts_rank_cd of the terms OR-ed |
search.partial.term_bonus |
1.0 |
+ term_bonus * terms the Record holds) |
search.partial.candidates |
100 |
partial matches ranked per search |
search.semantic
| Setting |
Default |
Description |
search.semantic.model |
BAAI/bge-small-en-v1.5 |
embedding model searched: the name Scribe stores vectors under (inference.embeddings.model; fake:<model> for the fake provider) |
search.semantic.iterative_scan |
auto |
Vector search under filters (Kinds, Traits, lens, where) keeps its limit (design 8.8): auto: hnsw.iterative_scan = relaxed_order with pgvector >= 0.8 (see /readyz); off: over-fetch by overfetch_factor instead |
search.semantic.max_scan_tuples |
20000 |
hnsw.max_scan_tuples of an iterative scan; past it the response says semantic_truncated |
search.semantic.overfetch_factor |
4 |
the index returns candidates * factor nearest vectors of the searched Kinds before the Records' filters apply; it also sets hnsw.ef_search (capped at 1000), which bounds recall |
search.hybrid
Reciprocal rank fusion: w / (rrf_k + rank) per list
| Setting |
Default |
Description |
search.hybrid.rrf_k |
60 |
|
search.hybrid.lexical_weight |
1.0 |
|
search.hybrid.semantic_weight |
1.0 |
|
search.resolve
POST /resolve (design 8.9): words of a text to candidate Kinds, Attributes and indexed values (store.values). Phrases of 1..max_phrase_words words, longest first; levels in order, a phrase stopping at the first that matches.
search.resolve.results
| Setting |
Default |
Description |
search.resolve.results.default_limit |
20 |
|
search.resolve.results.max_limit |
100 |
|
search.resolve (continued)
| Setting |
Default |
Description |
search.resolve.max_text_chars |
500 |
|
search.resolve.max_words |
40 |
words of the text considered |
search.resolve.max_phrase_words |
3 |
|
search.resolve.min_word_chars |
2 |
shorter words are never matched alone |
search.resolve.trigram_min_chars |
3 |
shortest phrase matched by similarity |
search.resolve.trigram_threshold |
0.45 |
least trigram similarity of a similar match |
search.resolve.per_phrase |
5 |
similar values kept per phrase |
search.resolve.exact_per_phrase |
20 |
exact value matches read per phrase form |
search.resolve.separators |
",;:!?()[]{}\"" # split words besides blanks; others stay in words (C++, S/M) |
|
search.resolve.edges |
".'" |
stripped from both ends of each word |
search.resolve.plural_rules |
[["s", ""], ["es", ""]] |
(plural, singular) suffixes folded both ways |
search.resolve.embedding_records |
10 |
nearest Records read by the embedding level |
search.resolve.levels |
{exact: true, synonym: true, trigram: true, embedding: true} |
|
search.resolve.scores |
{exact: 1.0, synonym: 0.95, trigram: 0.85, embedding: 0.7, description: 0.6} |
|
search.resolve.misses
Recording of what matched nothing (store.resolve_misses)
| Setting |
Default |
Description |
search.resolve.misses.permission |
data.read |
needed Domain-wide for a caller's misses to be kept |
search.resolve.misses.record_below_score |
null |
also keep phrases whose best candidate scores below this |
search.resolve.misses.queue_size |
10000 |
distinct phrases held in memory between writes |
search.resolve.misses.max_new_per_caller |
100 |
new phrases one caller adds per Domain between writes |
search.resolve.misses.flush_interval_s |
5.0 |
longest wait in memory before a batch write |
search.resolve.misses.write_timeout_s |
5.0 |
|
agents
Agent-facing reads (docs/agents.md). Context packs are also bounded by read.traverse; nothing here calls a model.
agents.context
POST /domains/{d}/context
| Setting |
Default |
Description |
agents.context.default_budget_tokens |
4000 |
estimated token budget when the request gives none |
agents.context.max_budget_tokens |
32000 |
|
agents.context.chars_per_token |
4.0 |
estimate: tokens = ceil(characters / chars_per_token) |
agents.context.default_depth |
1 |
neighborhood depth when the request gives none |
agents.context.max_depth |
2 |
|
agents.context.max_seeds |
20 |
seed Records of one pack |
agents.context.max_neighbors |
200 |
Records around the seeds |
agents.context.max_links |
500 |
|
agents.context.recent_changes_max |
50 |
change entries read before trimming, newest first |
agents.context.trim_order |
[lineage, recent_changes, far_neighbor_values, far_neighbors, schema_descriptions, near_neighbor_values, near_neighbors, schema] |
sections trimmed, in this order, to fit the budget |
agents.explain
GET /domains/{d}/kinds/{kind}/records/{key}/explain
| Setting |
Default |
Description |
agents.explain.max_links |
100 |
Links listed with their origin |
agents.explain.max_proposals |
20 |
Proposals that changed the Record |
agents.feed
GET .../branches/{b}/events (SSE) and .../changes
| Setting |
Default |
Description |
agents.feed.poll_interval_s |
1.0 |
how often an open stream looks for new events |
agents.feed.heartbeat_s |
15.0 |
comment line after this much silence |
agents.feed.max_connection_s |
300.0 |
streams close after this long; clients resume with Last-Event-ID |
agents.feed.reauthorize_interval_s |
60.0 |
open streams check credential and grants again this often and close when refused (grants seen after keeper.auth.authz cache) |
agents.feed.retry_ms |
3000 |
reconnection delay advised to clients |
agents.feed.batch_events |
100 |
events read per poll |
agents.feed.changed_keys_max |
100 |
changed Record keys listed per event |
agents.feed.changes
Pages of GET .../changes
| Setting |
Default |
Description |
agents.feed.changes.default_limit |
50 |
|
agents.feed.changes.max_limit |
500 |
|
agents.feed (continued)
| Setting |
Default |
Description |
agents.feed.max_scan |
10000 |
first-parent Revisions walked back to find since |
webhooks
Webhooks (design 10.5): managed through Keeper, delivered from the outbox by lorebase_store.webhooks.deliver_pending (run periodically by a worker).
| Setting |
Default |
Description |
webhooks.allowed_schemes |
[https] |
URL schemes a webhook may use |
webhooks.allow_private_targets |
false |
refuse loopback, private and link-local targets |
webhooks.max_per_domain |
20 |
|
webhooks.secret_bytes |
32 |
random bytes of a new signing secret |
webhooks.encryption_key |
null |
secret://... (32+ characters) encrypting signing keys at rest; same value for Keeper and Scribe; null: Keeper refuses to create webhooks (compose and Helm supply one) |
webhooks.allow_unsealed_keys |
false |
local development only: without encryption_key, store new signing keys usable instead of refusing |
webhooks.previous_encryption_keys |
[] |
earlier keys, tried to open; keys are re-encrypted under the new |
webhooks.signature_header |
Lorebase-Signature |
t=<unix>,v1=<hex HMAC-SHA256 of "<t>.<body>"> |
webhooks.event_header |
Lorebase-Event |
|
webhooks.delivery_header |
Lorebase-Delivery |
|
webhooks.idempotency_header |
Idempotency-Key |
evt_<sequence>, the same on every attempt of an event |
webhooks.masked_value |
"***" |
sensitive values a webhook is not cleared for |
webhooks.erased_value |
"[erased]" |
vault values erased before a cleared webhook receives them |
webhooks.timeout_s |
10.0 |
time budget of one attempt as a whole (lookup, connects, status) |
webhooks.connect_timeout_s |
5.0 |
connecting to one address; then the next checked one (<= timeout_s) |
webhooks.max_attempts |
8 |
then the delivery is marked dead |
webhooks.backoff_base_s |
30.0 |
delay after the first failure ... |
webhooks.backoff_factor |
2.0 |
... multiplied by this after each next one ... |
webhooks.backoff_max_s |
3600.0 |
... up to this |
webhooks.lease_s |
120.0 |
a claimed delivery is retried by another worker after this; a pass running longer is abandoned (> pass_timeout_s + timeout_s) |
webhooks.batch |
100 |
events and deliveries per worker pass |
webhooks.concurrency |
8 |
attempts of one pass in flight at the same time |
webhooks.pass_timeout_s |
60.0 |
no attempt starts after this; the rest go to the next pass |
webhooks.changed_keys_max |
100 |
changed Record keys listed per event payload |
webhooks.retention_s |
604800.0 |
finished deliveries kept (7 days; events: store.outbox) |
vault
Personal data vault (design 13.5; docs/threat-model.md): values of pii Attributes with vault: true are stored encrypted here, Records hold tokens, and lorebase admin erase erases them on every Revision. Keeper, Scribe and every committing process need the same keys.
| Setting |
Default |
Description |
vault.token_key |
null |
secret://... (32+ characters) keying tokens and value fingerprints; never change it once values are stored |
vault.encryption_key |
null |
secret://... (32+ characters) encrypting new values |
vault.key_id |
"1" |
id stored with values sealed under encryption_key |
vault.previous_keys |
{} |
earlier encryption keys by id, still opened |
herald
Herald, the MCP server (design 11.2; docs/mcp.md). Short variables set by MCP host configurations: LOREBASE_URL (keeper.base_url), LOREBASE_API_KEY (keeper.api_key), LOREBASE_DOMAIN (domain), LOREBASE_BRANCH (branch).
| Setting |
Default |
Description |
herald.transport |
stdio |
stdio or streamable_http |
herald.server_name |
lorebase |
|
herald.domain |
null |
Domain used when a call names none |
herald.branch |
null |
null: the Domain's default branch |
herald.log_level |
WARNING |
log level of the MCP SDK |
herald.keeper
| Setting |
Default |
Description |
herald.keeper.base_url |
http://127.0.0.1:8080 |
|
herald.keeper.api_prefix |
/api/v1 |
|
herald.keeper.api_key |
null |
stdio only: the agent's key, e.g. secret://env/AGENT_API_KEY |
herald.keeper.timeout_s |
30 |
|
herald.keeper.max_retries |
2 |
resends after 429, 503 or no connection; 0 turns retries off |
herald.keeper.retry_max_wait_s |
5 |
longest pause before a retry; Retry-After is capped to it |
herald.keeper.verify_tls |
true |
|
herald.keeper.trust_env |
false |
false: proxy variables ignored; true: HTTP(S)_PROXY, NO_PROXY honored |
herald.keeper.user_agent |
lorebase-herald |
|
herald.http
Streamable_http: each caller sends its own key
| Setting |
Default |
Description |
herald.http.host |
127.0.0.1 |
|
herald.http.port |
8090 |
|
herald.http.path |
/mcp |
|
herald.http.stateless |
true |
|
herald.http.json_response |
true |
|
herald.http.allowed_hosts |
[] |
e.g. [herald.example.com]; empty: SDK default for the host |
herald.http.allowed_origins |
[] |
|
herald.http.credential_header |
Authorization |
|
herald.http.allow_anonymous |
false |
true: calls without a credential go to Keeper without one; local stacks with keeper.auth.mode none only (Keeper decides) |
herald.output
| Setting |
Default |
Description |
herald.output.default_format |
text |
text (compact) or json |
herald.output.max_chars |
12000 |
longer outputs are cut with a [truncated ...] marker |
herald.output.max_items |
50 |
|
herald.output.max_value_chars |
240 |
|
herald.output.max_issues |
20 |
|
herald.context
Lorebase_context packs
| Setting |
Default |
Description |
herald.context.token_budget |
3000 |
|
herald.context.max_token_budget |
20000 |
|
herald.context.depth |
1 |
|
herald.context.max_seeds |
5 |
|
herald.context.search_seeds |
3 |
|
herald.neighborhood
| Setting |
Default |
Description |
herald.neighborhood.default_depth |
1 |
|
herald.neighborhood.max_depth |
3 |
|
herald.neighborhood.default_max_nodes |
50 |
|
herald.neighborhood.max_nodes |
500 |
|
herald.neighborhood.max_edges |
1000 |
|
herald.neighborhood.max_seeds |
20 |
|
herald.changes
| Setting |
Default |
Description |
herald.changes.page_size |
50 |
|
herald.changes.max_pages |
10 |
|
herald.trace
Agent trace headers on every Keeper call (keeper.trace)
| Setting |
Default |
Description |
herald.trace.enabled |
true |
false: send no session, tool or traceparent header |
herald.trace.session_header |
X-Lorebase-Session |
|
herald.trace.tool_header |
X-Lorebase-Tool |
|
herald.trace.trace_header |
X-Lorebase-Trace |
|
herald.trace.opt_in |
false |
true: send "<trace_header>: on" on every call |
herald.trace.session_id |
null |
session when the caller names none; null: MCP session id, else none (Keeper infers) |
guide
Guide, the optional exploration assistant (docs/guide.md): POST /embed and POST /explore. It holds no Keeper credential: every read carries the caller's own. Its chat model is the inference.llm model with the overrides of inference.llm.tasks.<agent.task>.
| Setting |
Default |
Description |
guide.host |
127.0.0.1 |
|
guide.port |
8095 |
|
guide.readiness_timeout_s |
3 |
/readyz probes Keeper's /healthz |
guide.data_dir |
~/.local/state/lorebase/guide |
Guide's own state files |
guide.mode_memory_file |
structured_output_modes.json |
remembered structured output modes; null: off |
guide.keeper
| Setting |
Default |
Description |
guide.keeper.base_url |
http://127.0.0.1:8080 |
|
guide.keeper.api_prefix |
/api/v1 |
|
guide.keeper.timeout_s |
30 |
|
guide.keeper.max_retries |
1 |
resends after 429, 503 or no connection; 0 turns retries off |
guide.keeper.retry_max_wait_s |
3 |
|
guide.keeper.verify_tls |
true |
Keeper is reached directly; proxy variables are ignored |
guide.keeper.user_agent |
lorebase-guide |
|
guide.http
| Setting |
Default |
Description |
guide.http.allowed_origins |
[] |
browser origins allowed (CORS and Origin check), e.g. [https://lorebase.example.com] |
guide.http.default_origins |
[] |
used while allowed_origins is empty (an explicit [] logs a warning); compose sets the web service's origins |
guide.http.credential_header |
Authorization |
|
guide.http.session_cookie |
"" |
Keeper's session cookie name (e.g. __Host-lorebase_session) when Guide is served under the web app's origin; empty: off |
guide.http.csrf_header |
X-CSRF-Token |
must hold the session's token; checked against Keeper |
guide.http.allow_anonymous |
false |
true only with a Keeper in auth mode none, on a local stack |
guide.http.max_body_bytes |
16384 |
|
guide.trace
Agent trace (docs/agent-trace.md)
| Setting |
Default |
Description |
guide.trace.enabled |
true |
session headers on every read, reasoning posted as notes |
guide.trace.session_header |
X-Lorebase-Session |
|
guide.trace.tool_header |
X-Lorebase-Tool |
|
guide.trace.opt_in_header |
X-Lorebase-Trace |
|
guide.trace.opt_in_value |
"on" |
empty: no opt-in header (Keeper's trace settings decide) |
guide.trace.notes |
true |
post reasoning as session notes (a write to Keeper's trace store with the caller's credential); false: headers only |
guide.trace.notes_path |
/trace/sessions/{session}/notes |
|
guide.trace.max_note_chars |
2000 |
|
guide.agent
A request may lower these budgets, never raise them
| Setting |
Default |
Description |
guide.agent.task |
explore |
inference.llm.tasks.explore overrides the chat model |
guide.agent.max_steps |
8 |
tool calls; then one last turn to answer |
guide.agent.max_tokens_per_call |
1024 |
completion cap of one model call |
guide.agent.max_total_tokens |
40000 |
input + output tokens of one exploration |
guide.agent.timeout_s |
120 |
wall clock of one exploration |
guide.agent.max_question_chars |
2000 |
|
guide.agent.max_result_chars |
6000 |
tool result text shown to the model; longer is cut |
guide.agent.max_value_chars |
300 |
|
guide.agent.max_items |
20 |
|
guide.agent.max_depth |
2 |
|
guide.agent.max_nodes |
60 |
|
guide.agent.max_context_tokens |
3000 |
|
guide.agent.max_answer_chars |
4000 |
|
guide.agent.temperature |
null |
null: the chat model's own (inference.llm.temperature) |
guide.limits
Per process; protects the model budget
| Setting |
Default |
Description |
guide.limits.max_concurrent |
4 |
|
guide.limits.max_concurrent_per_caller |
1 |
|
guide.limits.explorations_per_minute |
6 |
per credential |
guide.limits.embeds_per_minute |
120 |
per credential |
guide.limits.max_embed_chars |
2000 |
|
guide.limits.max_concurrent_verifications |
16 |
credential checks against Keeper at once; more: 429 |
guide.limits.max_tracked_callers |
10000 |
callers counted by the rate windows; full: new callers 429 |
scribe
Scribe, the job worker (docs/scribe.md): Source synchronization, webhook delivery, queued per-Attribute index DDL (store.index, as database.maintenance_dsn when set) and garbage collection. Job retries and stale locks: store.jobs.
scribe.worker
| Setting |
Default |
Description |
scribe.worker.concurrency |
2 |
jobs run at the same time by one process |
scribe.worker.poll_interval_s |
2.0 |
pause of an idle slot before it looks for jobs again |
scribe.worker.heartbeat_interval_s |
30.0 |
running jobs refresh their lock this often (<= stale_after_seconds / 2) |
scribe.worker.shutdown_grace_s |
30.0 |
on SIGTERM, running jobs get this long, then go back to the queue |
scribe.worker.shutdown_cancel_s |
5.0 |
then cancelled jobs get this long to stop before they are released anyway |
scribe.worker.outcome_retry_delay_s |
1.0 |
pause before the one more try at writing an outcome the store refused; failing again, the run counts unrecorded and the job runs again once its lock goes stale |
scribe.worker.liveness_file |
null |
file the job loop rewrites every liveness_interval_s with its live slots, running jobs and last store round trip, for a healthcheck or probe (python -m lorebase_scribe.liveness FILE); null writes none |
scribe.worker.liveness_interval_s |
10.0 |
how often the liveness file is rewritten |
scribe.worker.statement_timeout_ms |
0 |
Scribe's own connections; 0 disables (GC and index builds run long) |
scribe.worker.principal |
svc_scribe |
author of Scribe's commits and Proposals |
scribe.worker.types |
[] |
job types this process runs; empty: all of them |
scribe.scheduler
One process at a time is leader (Postgres advisory lock)
| Setting |
Default |
Description |
scribe.scheduler.enabled |
true |
|
scribe.scheduler.tick_s |
15.0 |
interval between scheduling passes |
scribe.scheduler.lock_name |
lorebase.scribe.scheduler |
hashed into the advisory lock key |
scribe.scheduler.webhook_interval_s |
10.0 |
webhook delivery pass (retries included); 0 disables |
scribe.scheduler.gc_schedule |
"0 3 * * *" |
cron (UTC) of the collection of every Domain; null disables |
scribe.scheduler.gc_dry_run |
false |
|
scribe.scheduler.source_schedules |
true |
queue Source synchronizations from each Source's schedule |
scribe.scheduler.embeddings_interval_s |
60.0 |
embedding pass of every Domain; 0 disables |
scribe.scheduler.value_index_interval_s |
300.0 |
value index pass of every Domain (store.values); 0 disables |
scribe.scheduler.coverage_interval_s |
300.0 |
coverage statistics of every Domain (store.coverage); 0 disables |
scribe.scheduler.consolidation_interval_s |
3600.0 |
misses -> vocabulary suggestions, while store.resolve_misses.enabled; 0 disables |
scribe.scheduler.audit_verify_interval_s |
86400.0 |
verification of every audit hash chain; 0 disables |
scribe.scheduler.domain_batch |
100 |
Domains read per query |
scribe.sources
Source synchronization (design 9.1)
| Setting |
Default |
Description |
scribe.sources.allowed_connections |
[] |
secret references a Source connection (or an http_json header value) may name, e.g. secret://env/ERP_DSN; empty: no Source can be synchronized |
scribe.sources.commit_batch |
5000 |
changes per Revision; one row's changes are never split |
scribe.sources.max_row_errors |
100 |
skipped rows tolerated; more fail the run without committing |
scribe.sources.error_samples |
20 |
row errors kept in the run record |
scribe.sources.auto_merge_protected |
true |
automatic merge policies also merge into protected branches (waives required_approvals, not steward or role rules) |
scribe.sources.csv
What an upstream change does to a value someone else set since the last sync is set per Source in the schema: local_edits: review (default: a Conflict), keep or source_wins.
| Setting |
Default |
Description |
scribe.sources.csv.allowed_roots |
[] |
directories local CSV files may be read from; empty: none |
scribe.sources.csv.allowed_hosts |
[] |
hosts CSV URLs may point at; empty: none |
scribe.sources.csv.allowed_schemes |
[https] |
|
scribe.sources.csv.allow_private_targets |
false |
true: URL hosts may resolve to private/loopback addresses |
scribe.sources.csv.max_bytes |
104857600 |
larger files or downloads are refused (100 MiB); reads stop here |
scribe.sources.csv.timeout_s |
60.0 |
time budget of one download, lookup included |
scribe.sources.csv.encodings |
[utf-8-sig, cp1252] |
tried in order |
scribe.sources.csv.delimiters |
",;\t\|" |
delimiters the sniffer may pick; .tsv files use a tab |
scribe.sources.csv.sniff_bytes |
65536 |
|
scribe.sources.json_files
Json and jsonl Sources (local files)
| Setting |
Default |
Description |
scribe.sources.json_files.allowed_roots |
[] |
directories JSON files may be read from; empty: none |
scribe.sources.json_files.max_bytes |
104857600 |
largest json document, parsed whole (100 MiB) |
scribe.sources.json_files.max_line_bytes |
16777216 |
longest jsonl line (16 MiB); jsonl files are streamed |
scribe.sources.parquet
Parquet Sources (local files; needs lorebase-scribe[parquet])
| Setting |
Default |
Description |
scribe.sources.parquet.allowed_roots |
[] |
directories Parquet files may be read from; empty: none |
scribe.sources.parquet.batch_rows |
10000 |
rows decoded per batch; files are streamed, never read whole |
scribe.sources.http_json
Http_json Sources (GET, JSON responses, pagination)
| Setting |
Default |
Description |
scribe.sources.http_json.allowed_hosts |
[] |
hosts URLs (next pages included) may point at; empty: none |
scribe.sources.http_json.allowed_schemes |
[https] |
|
scribe.sources.http_json.allow_private_targets |
false |
true: hosts may resolve to private/loopback addresses |
scribe.sources.http_json.timeout_s |
30.0 |
time budget of one request, lookup included |
scribe.sources.http_json.max_bytes |
52428800 |
larger responses are refused (50 MiB) |
scribe.sources.http_json.max_pages |
1000 |
more pages fail the run; a Source's http.max_pages may lower it |
scribe.sources.http_json.max_attempts |
4 |
attempts per request, the first included |
scribe.sources.http_json.retry_statuses |
[429, 500, 502, 503, 504] |
retried, as are network errors and timeouts |
scribe.sources.http_json.backoff_base_s |
1.0 |
pause before the first retry |
scribe.sources.http_json.backoff_factor |
2.0 |
each further retry waits this many times longer |
scribe.sources.http_json.backoff_max_s |
60.0 |
longest pause, a server's Retry-After included |
scribe.sources.postgres
| Setting |
Default |
Description |
scribe.sources.postgres.statement_timeout_ms |
60000 |
statement_timeout of the read-only transaction |
scribe.sources.postgres.connect_timeout_s |
10.0 |
|
scribe.sources.postgres.fetch_rows |
5000 |
rows fetched per round trip |
scribe.sources.mysql
MySQL and MariaDB Sources; needs lorebase-scribe[mysql]
| Setting |
Default |
Description |
scribe.sources.mysql.statement_timeout_ms |
60000 |
max_execution_time (MariaDB: max_statement_time, seconds) |
scribe.sources.mysql.connect_timeout_s |
10.0 |
|
scribe.sources.mysql.fetch_rows |
5000 |
rows fetched per round trip |
scribe.sources.mssql
SQL Server Sources; needs lorebase-scribe[mssql]
| Setting |
Default |
Description |
scribe.sources.mssql.statement_timeout_ms |
60000 |
query timeout and LOCK_TIMEOUT |
scribe.sources.mssql.connect_timeout_s |
10.0 |
login timeout |
scribe.sources.mssql.fetch_rows |
5000 |
rows fetched per round trip |
Documents into reviewable Proposals (docs/extraction.md)
| Setting |
Default |
Description |
scribe.extraction.tiers |
[0, 1] |
0 rules, 1 local span and relation models, 2 the chat model of inference.llm; add 2 once a model is configured |
scribe.extraction.auto_accept_threshold |
0.9 |
items at or above are accepted at once (above 1: none) |
scribe.extraction.min_confidence |
0.3 |
weaker candidates are dropped |
scribe.extraction.mode |
doubts_only |
ingestion mode of Domains without their own: review_all (nothing accepted at once) or doubts_only; direct is refused here, a Domain's settings.extraction may choose it |
scribe.extraction.domain_may_loosen |
[mode] |
members a Domain may set looser than these (mode, auto_accept_threshold, min_confidence); []: none |
scribe.extraction.direct_merge_protected |
true |
direct-mode jobs may merge into a protected branch (waives required_approvals, not steward or role rules); false: their Proposals wait for approvals |
scribe.extraction.new_record_kinds |
null |
Kinds that may get new Records; null: every natural-key Kind |
scribe.extraction.value_cue_words |
8 |
a rule-found value needs a cue word this close before it |
scribe.extraction.max_items_per_job |
5000 |
|
scribe.extraction.max_document_chars |
2000000 |
parsed text beyond this is not extracted |
scribe.extraction.prompt_version |
extract-v1 |
recorded in Lineage and the cache key; bump with prompts |
scribe.extraction.cache |
true |
reuse tier 2 answers for unchanged chunk, schema and model |
scribe.extraction.max_tokens_per_job |
2000000 |
tier 2 stops after this many tokens; 0: no limit |
Evidence check of new document versions and retirements
| Setting |
Default |
Description |
scribe.extraction.versions.max_values_per_job |
5000 |
values one document_version_check job looks at; the rest goes to a continuation job |
PDF and DOCX parsed in a child process without network
| Setting |
Default |
Description |
scribe.extraction.parser.isolated |
true |
false: in a worker thread, bounded by max_input_bytes only |
scribe.extraction.parser.timeout_s |
60.0 |
wall-clock time per document |
scribe.extraction.parser.cpu_s |
60 |
CPU seconds (RLIMIT_CPU) |
scribe.extraction.parser.memory_mb |
1024 |
address space in MiB (RLIMIT_AS, where enforced) |
scribe.extraction.parser.max_input_bytes |
52428800 |
larger documents are not parsed |
| Setting |
Default |
Description |
scribe.extraction.chunking.max_chars |
2000 |
largest chunk of parsed text |
scribe.extraction.chunking.overlap_chars |
200 |
repeated from the previous chunk (< max_chars / 2) |
Entity resolution against existing Records
| Setting |
Default |
Description |
scribe.extraction.matching.trigram_threshold |
0.45 |
least trigram similarity of a fuzzy title match |
scribe.extraction.matching.match_threshold |
0.6 |
least score of a match; below: new Record or dropped |
scribe.extraction.matching.ambiguity_margin |
0.05 |
candidates this close to the best make the match ambiguous |
scribe.extraction.matching.candidates |
5 |
candidates kept per mention |
scribe.extraction.matching.dictionary_max_titles |
20000 |
titles loaded per job for dictionary matching |
scribe.extraction.matching.min_title_chars |
4 |
shorter titles are not matched by dictionary |
Confidence given by each technique
| Setting |
Default |
Description |
scribe.extraction.confidence.key |
1.0 |
a key found by its pattern |
scribe.extraction.confidence.title |
0.95 |
an exact title |
scribe.extraction.confidence.normalized_title |
0.9 |
a title equal after case and punctuation folding |
scribe.extraction.confidence.value_max |
0.85 |
ceiling of rule-found values |
scribe.extraction.confidence.cooccurrence_max |
0.7 |
ceiling of Links from co-occurrence in a sentence |
scribe.extraction.confidence.heading |
0.6 |
Links between a heading's Record and those of its section |
scribe.extraction.confidence.tier2 |
0.8 |
facts from the chat model with verified evidence |
Schema suggestions: what the tiers see outside the schema
| Setting |
Default |
Description |
scribe.extraction.discovery.enabled |
true |
|
scribe.extraction.discovery.rules |
true |
tier 0: **Label:** fields, table columns, headings with codes |
scribe.extraction.discovery.span_labels |
[] |
tier 1: extra span labels suggesting Kinds, e.g. [certificate] |
scribe.extraction.discovery.relation_labels |
[] |
tier 1: extra relation labels suggesting Relations |
scribe.extraction.discovery.tier2 |
true |
tier 2: the model lists unlisted facts (verified evidence) |
scribe.extraction.discovery.llm_pass |
false |
a dedicated call per chunk through the schema_discovery route |
scribe.extraction.discovery.llm_max_items |
20 |
proposals of each list that call may return |
scribe.extraction.discovery.llm_max_tokens |
200000 |
token cap of that pass per job (0: none); apart from max_tokens_per_job |
scribe.extraction.discovery.llm_max_quote_chars |
300 |
proposals quoting more are discarded |
scribe.extraction.discovery.llm_max_quote_share |
0.5 |
... or quoting more than this share of the chunk |
scribe.extraction.discovery.llm_quote_share_min_chunk_chars |
400 |
... of chunks at least this long |
scribe.extraction.discovery.min_confidence |
0.3 |
weaker observations are dropped |
scribe.extraction.discovery.field_confidence |
0.6 |
a field line or table column |
scribe.extraction.discovery.heading_confidence |
0.5 |
a Kind named by a heading |
scribe.extraction.discovery.max_value_chars |
120 |
sample values are cut to this length |
scribe.extraction.discovery.vault_match_max_words |
12 |
longest quote word sequence compared with vault fingerprints |
scribe.extraction.discovery.vault_match_max_text_words |
400 |
longer quotes are not compared: evidence withheld |
scribe.extraction.discovery.vault_match_max_text_chars |
4000 |
same, by characters |
scribe.extraction.discovery.vault_match_max_candidates |
200000 |
word sequences compared per job; past it, withheld |
scribe.extraction.discovery.max_observations_per_job |
1000 |
|
scribe.embeddings
Embedding jobs (design 9.3); model: inference.embeddings
| Setting |
Default |
Description |
scribe.embeddings.page_rows |
500 |
Record rows read and rendered per query |
scribe.embeddings.batch_texts |
64 |
texts per call of the embedding model |
scribe.embeddings.max_records_per_job |
20000 |
rows one job reads; the rest goes to a continuation job |
scribe.embeddings.max_calls_per_second |
0.0 |
model calls per second of one job; 0: no limit |
scribe.embeddings.max_text_chars |
8000 |
rendered texts are cut to this length before hashing |
scribe.embeddings.template_cache_size |
64 |
schemas whose resolved templates are kept, by schema hash |
scribe.embeddings.retry
Retries of a failed call, after the provider's own retries
| Setting |
Default |
Description |
scribe.embeddings.retry.attempts |
2 |
attempts per call, the first included |
scribe.embeddings.retry.backoff_s |
2.0 |
pause before the first retry; doubles each time |
scribe.embeddings.retry.backoff_max_s |
60.0 |
longest pause |
scribe.consolidation
Consolidation of resolution misses into vocabulary suggestions (design 8.10), also run by lorebase-scribe consolidate DOMAIN. Suggestions are reviewed like any other; nothing changes the schema without an accepted Proposal.
| Setting |
Default |
Description |
scribe.consolidation.min_misses |
3 |
misses a phrase needs before it is considered |
scribe.consolidation.min_callers |
2 |
distinct callers it needs (1 for a single-agent installation) |
scribe.consolidation.max_phrases |
200 |
phrases per pass, the most missed first |
scribe.consolidation.value_similarity |
0.4 |
least trigram similarity to an indexed value (value alias) |
scribe.consolidation.name_similarity |
0.75 |
least name similarity to a Kind or Attribute (alias) |
scribe.consolidation.values_per_phrase |
3 |
|
scribe.consolidation.llm
Optional description proposals, task schema_consolidation;
| Setting |
Default |
Description |
scribe.consolidation.llm.enabled |
false |
the model sees names and missed phrases, never values |
scribe.consolidation.llm.min_misses |
10 |
|
scribe.consolidation.llm.max_attributes |
5 |
|
scribe.consolidation.llm.max_phrases |
20 |
|
scribe.consolidation.llm.max_description_chars |
500 |
|
scribe.consolidation.llm.max_tokens_per_pass |
20000 |
|
scribe.audit_checkpoints
Signed heads of the audit chains published outside the database; python -m lorebase_keeper audit verify --checkpoints then detects entries cut from a chain's end
| Setting |
Default |
Description |
scribe.audit_checkpoints.enabled |
false |
|
scribe.audit_checkpoints.interval_s |
3600.0 |
newer entries are not covered until the next checkpoint |
scribe.audit_checkpoints.streams |
[actions, reads] |
audit_log and audit_read_log |
scribe.audit_checkpoints.signing_key |
null |
Ed25519 PEM, e.g. secret://file//run/secrets/checkpoint.pem (openssl genpkey -algorithm ed25519); verifiers need only the public key (openssl pkey -pubout) |
scribe.audit_checkpoints.key_id |
main |
recorded in each checkpoint |
scribe.audit_checkpoints.prefix |
lorebase-audit/ |
object names: {prefix}{stream}/{UTC time}.json |
scribe.audit_checkpoints.sink |
file |
file | http | s3 |
scribe.audit_checkpoints.directory |
null |
file: a directory the database host cannot write |
scribe.audit_checkpoints.http
Http: PUT or POST of each checkpoint
| Setting |
Default |
Description |
scribe.audit_checkpoints.http.url |
null |
{name} is replaced by the object name |
scribe.audit_checkpoints.http.method |
PUT |
|
scribe.audit_checkpoints.http.headers |
{} |
values may be secret references |
scribe.audit_checkpoints.http.allowed_schemes |
[https] |
|
scribe.audit_checkpoints.http.allow_private_targets |
false |
off: global unicast addresses only, pinned, no redirects |
scribe.audit_checkpoints.http.timeout_s |
10.0 |
|
scribe.audit_checkpoints.s3
S3: PUT signed with Signature Version 4; use a bucket with versioning or object lock and PutObject-only credentials
| Setting |
Default |
Description |
scribe.audit_checkpoints.s3.endpoint |
null |
e.g. https://s3.eu-west-1.amazonaws.com |
scribe.audit_checkpoints.s3.bucket |
null |
|
scribe.audit_checkpoints.s3.region |
us-east-1 |
|
scribe.audit_checkpoints.s3.path_style |
true |
endpoint/bucket/key; false: bucket.endpoint/key |
scribe.audit_checkpoints.s3.access_key_id |
null |
secret references, e.g. secret://env/CHECKPOINT_KEY_ID |
scribe.audit_checkpoints.s3.secret_access_key |
null |
|
scribe.audit_checkpoints.s3.session_token |
null |
|
scribe.audit_checkpoints.s3.allowed_schemes |
[https] |
|
scribe.audit_checkpoints.s3.allow_private_targets |
false |
|
scribe.audit_checkpoints.s3.timeout_s |
10.0 |
|
scribe.reconciliation
Contradictions between Sources (design 10.6); a Domain overrides any member in its reconciliation setting
| Setting |
Default |
Description |
scribe.reconciliation.enabled |
true |
false: no candidate is raised |
scribe.reconciliation.stages |
[identity, values, normalize, semantic] |
run in this order; leave one out to disable it |
scribe.reconciliation.hold_auto_accept |
true |
extraction items raising a contradiction stay pending |
scribe.reconciliation.sweep_interval_s |
0.0 |
scheduled sweep of every Domain; 0 disables it |
scribe.reconciliation.sweep_max_changes_per_job |
50000 |
history changes one sweep job reads; then a continuation |
scribe.reconciliation.max_claims_per_candidate |
20 |
newest claims kept per candidate; the others are counted |
scribe.reconciliation.merge_check |
warn |
Proposal check of open candidates on the Proposals Scribe merges: off, warn or block; the stricter of this and the Domain reconciliation_check applies (off < warn < block), so a Domain can tighten it, never loosen it |
scribe.reconciliation.triggers
| Setting |
Default |
Description |
scribe.reconciliation.triggers.extraction |
true |
check extraction value items while they are assembled |
scribe.reconciliation.triggers.source_sync |
true |
check a Source sync's changes before they are merged |
scribe.reconciliation.triggers.document_versions |
true |
check evidence quotes when a document gets a new version or is retired (stale candidates, signal evidence_gone) |
scribe.reconciliation.duplicates
| Setting |
Default |
Description |
scribe.reconciliation.duplicates.trigram_threshold |
0.8 |
least title similarity of a duplicate candidate |
scribe.reconciliation.duplicates.max_pairs_per_kind |
1000 |
duplicate pairs examined per Kind and pass |
scribe.reconciliation.normalize
String and text values; never changes a stored value
| Setting |
Default |
Description |
scribe.reconciliation.normalize.units_file |
null |
YAML table of units replacing the one shipped with lorebase_core |
scribe.reconciliation.normalize.numeric_tolerance |
0.0 |
relative difference under which numbers are equal |
scribe.reconciliation.normalize.max_quantity_length |
64 |
longer values are not read as a number and a unit |
scribe.reconciliation.normalize.negation_words |
[not, "no", never, without] |
equal without them: marked negation |
scribe.reconciliation.semantic
Embeddings of competing text values (needs inference.embeddings)
| Setting |
Default |
Description |
scribe.reconciliation.semantic.enabled |
false |
true also needs the stage in stages |
scribe.reconciliation.semantic.equivalence_threshold |
null |
similarity above which paraphrases are not raised; null: all |
scribe.reconciliation.semantic.equivalent_similarity |
0.9 |
least similarity of text values marked equivalent (ranked last) |
scribe.reconciliation.semantic.duplicate_threshold |
0.92 |
least similarity of stored Record vectors for a duplicate |
scribe.reconciliation.semantic.judge |
false |
ask the chat model whether two texts contradict |
scribe.reconciliation.stale
| Setting |
Default |
Description |
scribe.reconciliation.stale.max_age |
null |
ISO 8601 duration (e.g. P30D); null: off. Per Kind and Attribute in the Domain settings |
scribe.reconciliation.stale.source_unseen |
true |
raise Records an incremental Source has not read within its stale_after (a Source setting in the schema; null there: off) |