Skip to content
Keelmint

Notifications

The app tells users about changes to their account in-app: a bell in the header and a /notifications page, updated live on every open tab and on every API instance. This page covers:

  1. How it works (creation, the stream, fan-out between instances).
  2. How to add a notification type.
  3. Failure modes, and what the user sees.
  4. What production needs (reverse proxy, HTTP/2, PgBouncer).
  5. How to try it locally and in the tests.

Related docs:

  • Background jobs: the mail queue the password-changed notice goes through.

1. How it works

The flow

PATCH /api/members/:userId/roles (or register, accept invitation, change or reset password)
   |
   +-> one Prisma transaction
         |- the business write (+ its audit event)
         |- INSERT notifications (type, data)              NotificationsService.notify(tx, ...)
         |- SELECT pg_notify('notifications', '{"userId":"…","id":"…"}')
         '- password only: INSERT the mail.send job        MailQueue.enqueue(tx, ...)
   |
   commit   -> Postgres delivers the NOTIFY to every LISTENing connection
   rollback -> no row, no NOTIFY, no mail


every API instance: NotificationListener (one pg connection: LISTEN notifications)
   '- NotificationStreamHub.dispatch(userId, id)
         |- no stream for that user on this instance? nothing
         '- event: notification {id}, then event: unread-count {count}, to each of the user's streams


browser: NotificationsStore (HttpClient with fetch streaming)
   |- unread-count -> the bell's badge (and a list reload when already live: another tab changed rows)
   '- notification -> lists reload (the bell's menu if open, the /notifications page)

There is no in-process shortcut: even on a single instance, an event reaches a stream only through Postgres. One instance and many therefore behave the same.

Marking read, read-all and dismissing (not reading) also send a pg_notify, with {"userId":"…"} and no id. Every open tab of that user then gets a fresh unread-count.

The pieces

PieceWhereJob
Notification modellibs/core/db/prisma/core/notification.prismauserId, type, data, readAt, plus the base fields. Dismissing is a soft delete. No text is stored.
NOTIFICATION_TYPES, NotificationDataByTypelibs/core/shared/src/lib/notification.tsThe contract: the union, the data of each type, and the stream event names.
NotificationsService.notify@core/api-notificationsWrites the row and runs pg_notify, in the caller’s transaction.
NotificationsQueryService, the controllersameList, count, read, read-all and dismiss, for the caller’s own notifications only.
NotificationStreamHubsameThis instance’s open streams: heartbeat, expiry and fan-out.
NotificationListenersameOne LISTEN connection, kept alive and reconnected with backoff.
NotificationsStore@core/notifications-data-accessThe stream, reconnect, the polling fallback and the unread count.
NotificationsBell, NotificationsPage@core/notifications-featureThe UI, in el and en.

Notifications belong to the user, not to an organization: the bell shows the same list whichever organization is active. A notification about organization data carries the organization’s id in data (ROLES_CHANGED does), so a client can switch to it before opening the linked page. A platform admin never switches (they have no active organization): the link opens directly.

The types today:

TypeRecipientWritten bydata
WELCOMEthe new userregister, accept an invitation with a new account{}
ROLES_CHANGEDthe member whose roles movedPATCH /members/:userId/roles, only when added or removed is not empty{ added, removed, organizationId } (each entry a RoleRef plus the unit for a unit-scoped one)
INVITATION_ACCEPTEDthe inviteraccept invitation (new or existing account){ email, userId, organizationId }
PASSWORD_CHANGEDthe userprofile password change, password reset (both also send mail){ via: 'change' | 'reset' }
TWO_FACTOR_ENABLEDthe userturning 2FA on in the profile (also mailed){ via: 'self' | 'admin' }
TWO_FACTOR_DISABLEDthe userturning 2FA off, or a platform admin’s 2FA reset (also mailed){ via: 'self' | 'admin' }
TWO_FACTOR_RECOVERY_CODE_USEDthe usera login with a recovery code (also mailed){ remaining } (unused codes left)
SIGN_IN_METHOD_ADDEDthe userlinking Google to the account (also mailed){ provider }
SIGN_IN_METHOD_REMOVEDthe userunlinking Google (also mailed){ provider }
HELPDESK_TICKET_CREATEDstaff (platform admins), not the reportercreate ticket{ ticketId, number, subject, category }
HELPDESK_REPORTER_REPLIEDstaff who took part, else all staffreporter reply{ ticketId, number, subject }
HELPDESK_STAFF_REPLIEDthe reporter (also mailed)public staff reply{ ticketId, number, subject }
HELPDESK_STATUS_CHANGEDthe reporter, unless they changed it (also mailed)staff status change{ ticketId, number, subject, status }
TESTthe callerPOST /api/notifications/test (development only, see Local setup and tests){ sentAt }

The stream

GET /api/notifications/stream, with the normal Authorization: Bearer header, answers text/event-stream:

event: unread-count
data: {"count":2}


event: notification
data: {"id":"9b2f6c1e-…"}


: heartbeat


event: token-expired
data: {}
  • unread-count: sent on connect, and after every change: a new notification, read, read-all or dismiss, including changes made in another tab.
  • notification: sent for a new notification. An unread-count follows.
  • : heartbeat: an SSE comment, every 25 s.
  • token-expired: sent when the stream closes, one second after the access token expires (exp + 1 s). The extra second matters: JWT exp has one-second granularity, so the client’s reconnect presents a token the guard rejects, and the interceptor turns the 401 into a refresh and a retry.
  • Errors: a missing or expired token is a normal JSON 401, because the guards run before any stream header. More than 10 streams for one user on one instance answer 429 (JSON, the same way).
  • No replay: there is no Last-Event-ID support. After a reconnect, the first unread-count and a list reload bring the client up to date.
  • Never an error event: the server never turns a failure into an SSE error event, because a database error can carry row values. Failures are logged and the event is skipped.

The browser reads the stream with fetch streaming: Angular’s HttpClient with withFetch() and reportDownloadProgress, so the bearer interceptor and the refresh-and-retry apply. It does not use EventSource, which cannot send an Authorization header.

LISTEN/NOTIFY

  • Channel and payload. Channel notifications. The payload is {"userId":"…","id":"…"} for a new notification, or {"userId":"…"} for a count-only change. It carries ids only: no text, no data, nothing a subscriber could not look up. pg_notify runs through tx.$executeRaw, because it returns void.
  • One connection per instance. The listener owns one dedicated pg client, outside Prisma’s pool, because LISTEN needs a session of its own. It is the only place in the libraries that imports pg as a value (listen-client.ts).
  • Dead sockets. The client sets TCP keepalive (first probe after 10 s), and the listener sends SELECT 1 every 30 s with a 5 s timeout. An idle LISTEN socket says nothing when it dies, so without the ping a half-open connection could go unnoticed for hours. Connect and LISTEN each time out after 5 s.
  • Backoff. A lost connection is replaced after 1, 2, 4, 8 and 16 s, then every 30 s. The backoff starts over only after a connection stayed up for 60 s, so one that drops right after LISTEN keeps backing off.
  • Resync. After a reconnect, the hub sends a fresh unread-count to every user with a stream on this instance, 10 queries at a time. A failed count is retried once.
  • Boot. The first LISTEN runs in onApplicationBootstrap. With no database, the API does not boot.
  • Shutdown (SIGTERM). In beforeApplicationShutdown, the hub ends every stream, and the listener closes its connection (and destroys the socket if end() takes more than 2 s). Without that, the HTTP server would wait for the streams forever. Clients reconnect, with backoff and jitter, to an instance that is still up.
  • Logs. The listener logs at warn when it loses the connection or a reconnect fails, and at log when the connection is back.

The client

  • Start and stop. The bell renders only while the user is signed in (the dashboard layout loads it lazily). It starts the store when it appears and stops it when it goes. stop() clears the count, and drops the results of any request from the earlier session.
  • Reconnect. After the stream ends, the store reconnects after 1, 2, 4 … 30 s, each jittered between half and all of that step, so tabs that lost the same server do not all return together. The backoff resets once a stream has stayed live for 60 s, as on the server.
  • Polling. While the stream is down, the store polls GET /api/notifications/unread-count every 60 s (and once at once).
  • Dead connections. 60 s without a byte (more than two heartbeats) counts as a dead connection: the store drops it and reconnects.
  • token-expired. The store reconnects at once, with no backoff.
  • End of session. If the reconnect gets a 401 even after the interceptor’s refresh, the session is over: the store stops, and the auth flow signs the user out.
  • Stability and PendingTasks. The open stream is an HttpClient request, so it holds an Angular PendingTasks entry for as long as the user is signed in. The app therefore never reports stable (ApplicationRef.isStable, whenStable()). That matters for SSR, for the service worker’s registerWhenStable strategy, and for whenStable() in tests: any spec that renders the bell must mock NotificationsStore.
  • Bundle. The bell and the page are lazy-loaded. Neither is in the initial bundle, and nothing outside the lib imports @core/notifications-feature statically.

2. Adding a notification type

The example: TODO_SHARED, sent to a user someone shared a todo with.

  1. The type and its data, in libs/core/shared/src/lib/notification.ts:

    export const NOTIFICATION_TYPES = [..., 'TODO_SHARED'] as const;
    

    export interface NotificationDataByType {
      ...
      TODO_SHARED: { todoId: string; title: string; byEmail: string };
    }

    Use plain JSON only, and never a secret: the owner can read the row. No migration is needed, because type is a string column.

  2. The notify() call, inside the transaction that makes it true, after the business write and its audit event:

    await this.notifications.notify(tx, recipientId, 'TODO_SHARED', {
      todoId: todo.id,
      title: todo.title,
      byEmail: caller.email,
    });
    • Import NotificationsModule into the feature module, and inject NotificationsService. The compiler checks data against the type.
    • Add a unit test that the call happens with tx, and that a failing notify fails the transaction.
    • If the type also needs an email, add a MailJob template and call mailQueue.enqueue(tx, …) next to notify. See password-changed for an example.

  3. The text, in el and en. In libs/core/client/i18n/src/assets/i18n/{en,el}.json, under notifications.types, add:

    "TODO_SHARED": "{{email}} shared “{{title}}” with you."

    Then add a case to notificationText() (libs/core/client/notifications-feature/src/lib/utils/notification-text.ts) that returns that key and its params. It must stay total: read data defensively, as the existing cases do, because a row can come from a newer or older build. Its default case returns the generic notifications.types.unknown (“You have a new notification.”), which is also what an older client shows for a type it does not know. translations.spec fails until both languages have the key with the same {{params}}.

  4. The click route. Add a line to notificationLink() (notification-link.ts), for example TODO_SHARED: '/todos'. Today ROLES_CHANGED and PASSWORD_CHANGED go to /user-profile, and INVITATION_ACCEPTED to /members (of the organization in data.organizationId). Without a line, a click only marks the notification read. A notification whose data.organizationId is not the active organization switches to it first (injectOpenNotification, notification-open.ts, used by the bell and /notifications); a refused switch (no longer a member, suspended) is toasted and goes nowhere. For a platform admin nothing switches: the link opens as it is, and an /organization… link opens that organization’s screen (/organizations/<id>…).

  5. The icon. Add the type to ICONS in notification-icon.ts, which picks the lucide icon the bell’s panel shows. The map covers every type, so the build fails until the new one is there. A new icon also goes into NOTIFICATION_ICONS. A type an older client does not know gets the bell.

3. Failure modes

What breaksWhat happens
notify fails (a DB error)The caller’s transaction rolls back: no change, no notification. The request fails as it would on any DB error.
The LISTEN connection dropsNew notifications are still written. Streams on that instance get nothing until it reconnects (1–30 s), then they get a fresh count. A notification created during the gap is reflected only through that count resync: the badge corrects, and the lists show it when they next load.
An API instance stopsIts streams end. Clients reconnect to another instance, with backoff and jitter, and poll the count meanwhile.
A proxy buffers or cuts the streamThe client’s 60 s stall watchdog reconnects. Polling keeps the badge roughly right.
The access token expiresThe stream ends with token-expired, and the client reconnects with a refreshed token.
The session is over (refresh fails)The stream stops for good, and the auth flow signs the user out.
A user opens many tabsUp to 10 streams per user per instance. The 11th answers 429, and that tab polls the count.

4. Production

  • Reverse proxy. SSE needs buffering off, and a read timeout above the 25 s heartbeat. Nest already sends X-Accel-Buffering: no and Cache-Control: … no-transform. An nginx example:

    location /api/notifications/stream {
      proxy_pass http://api;
      proxy_http_version 1.1;
      proxy_set_header Connection '';
      proxy_buffering off;
      proxy_cache off;
      proxy_read_timeout 1h;
      gzip off;
    }

    Never compress text/event-stream anywhere on the path.

  • HTTP/2. Over HTTP/1.1, every open tab holds one of the browser’s roughly 6 connections per origin. Serve HTTP/2 at the proxy.

  • PgBouncer. LISTEN needs a session. In transaction pooling mode it silently receives nothing. Point DATABASE_URL at Postgres directly, or at a session-mode pool.

  • LISTEN gaps. Notifications created while an instance’s LISTEN connection is down are only reflected through the count resync after it reconnects. That is an accepted limit for v1.

  • Scale. Every instance receives every NOTIFY. For each one, it runs one unread-count query per affected user that has a stream on that instance. That is fine to start with. A busy deployment would batch the counts, or carry the count in the payload.

  • Sessions. An open tab keeps the session alive: each time the access token expires (15 min by default, JWT_ACCESS_TTL), the stream reconnects through the refresh flow. The refresh token’s rotation and lifetime still apply, but a tab left open never goes idle on the server’s side.

5. Local setup and tests

There is nothing to configure: the API LISTENs on the same DATABASE_URL. To see it work:

  1. Run pnpm nx serve keelmint-api and pnpm nx serve keelmint-client.
  2. Sign in as a user.
  3. In a second browser, sign in as a platform admin and change the first user’s roles.
  4. The bell updates without a reload.

To watch the raw stream:

Terminal window
curl -N -H "Authorization: Bearer $ACCESS_TOKEN" http://localhost:3000/api/notifications/stream

The tests:

  • pnpm nx test core-api-notifications: notify, the queries, the hub and the listener (with fake timers).
  • pnpm nx run-many -t test -p core-client-notifications-data-access core-client-notifications-feature: the parser, the store (reconnect, polling, stall), the bell and the page.
  • PORT=3001 pnpm nx e2e keelmint-api-e2e: notifications.spec.ts covers every trigger, the ownership 404s, and the live stream read with fetch. It also spawns a second API instance (port 3997, 3-second tokens), which receives an event triggered on the first and closes a stream at token expiry.

Testing live delivery in development

In development (isDevMode()), the bell’s panel has a Send test notification button in its footer. It calls POST /api/notifications/test, which writes a TEST notification for the signed-in user through the normal notify() path (row and pg_notify in one transaction). The new item then arrives through LISTEN, the hub and the stream like any other: the badge goes up and the open panel reloads. The panel stays open, so you can watch it arrive. Under the button, the panel says “Sent” or that sending failed. If it says “Sent” and nothing arrives, the stream is the problem, not the write.

  • The button. It shows only when the DEV_MODE token (@core/common-data-access) is true. That defaults to Angular’s isDevMode(), so a production build never shows it. Specs override the token.
  • The route. It needs a signed-in user, notifies only that user, and allows 10 calls a minute per user. It is a self route (@Authenticated()), so the permission catalog has nothing to grant.
  • Not in production. The controller lives in NotificationsDevModule. CoreModule imports it through ConditionalModule.registerWhen(NotificationsDevModule, devRoutesEnabled), which runs after the environment is loaded and validated. With NODE_ENV=production the route is never registered. It is absent from the OpenAPI document, and a call gets a plain 404 before any guard runs, even without a token. A guard that answered 404 on a registered route would still publish the route, and would rely on one check that a refactor could drop. The handler also throws a 404 when NODE_ENV is production, as a second line in case the module is ever imported directly. The e2e suite checks the 404 against its production instance (port 3997).
  • Set NODE_ENV=production in production. The route is registered whenever NODE_ENV is anything but production, and the env schema defaults NODE_ENV to development. A production deploy that leaves it unset therefore exposes the route. The shipped Dockerfile sets ENV NODE_ENV=production, but a deploy that does not use it must set the variable itself.

From a terminal:

Terminal window
curl -X POST -H "Authorization: Bearer $ACCESS_TOKEN" http://localhost:3000/api/notifications/test

6. The password-changed email

A password change or reset also queues a password-changed mail (el/en, according to the user’s language) in the same transaction, through MailQueue.enqueue. It says the password was changed and that other sessions were signed out. Its only link is ${APP_URL}/auth/forgot-password, a plain page that carries no token, so the mail holds nothing an attacker who reads it could use. If the transaction rolls back, there is no mail either.