History
84 changes · 29 decisions · since 2026-08-04
- 2026-08-29Shorter chart windows: 1D, 3D, 1W
- Asked for directly. Both history routes capped days at a minimum of 7 — the server already clamped it to 1 internally (MAX_HISTORY_DAYS in services/stocks.py), so the only thing standing in the way was the route's own Query(..., ge=7, ...), lowered to ge=1 on both /positions/{id}/history and /value/daily.
- Three new range pills ahead of 1M in RANGES (web/app/stocks/page.tsx): 1D, 3D, 1W. No other change needed -- a 1-point window already had a correct empty state ("Only one day recorded so far") from ADR-036, and a window with no settled days at all already had "No daily history found".
- 662 tests pass (up from 661), including a new HTTP-level test pinning that days=1/3/7 are accepted rather than refused.
- 2026-08-29Bring the hover total back, computed correctly this time
- The previous fix removed the misleading number entirely on a multi-series chart rather than compute a correct one. Asked for directly: "era giro aparecer o numero liquido do hover, desde que fosse o certo" — a real total was worth keeping, once it actually was one.
- Hovering now shows "Total <value>", computed from the portfolio's own combined series (totalOf(series), already used by Movement) read the same way any single line is — never by summing or averaging the individual holdings' own percentages, since 30% and −10% growth do not average to 20% of the combined value. Verified against the fixed since-start summary already shown below the chart: "Total −0.1%" on hover matched "▼1,57 € · −0.1%" exactly.
- The legend keeps showing each line's own figure for the same day, so the breakdown that motivated Change (€) in the first place is still right there next to the total that motivated this fix.
- 2026-08-29Fix: the hover number wasn't a total, but looked like one
- Caught live, on the deployed chart: hovering "By type" showed a single big number ("+194,61 €") sitting above a legend reading Bonds −4432,17 €, Stocks +194,61 €, Crypto +514,32 € — and the owner reasonably read the big number as some kind of net figure. It wasn't; it was just Stocks' own value, because the readout always showed series[0] regardless of how many series were on the plot, a leftover from when this component only ever drew one line.
- Fixed by removing the ambiguity rather than trying to compute a "correct" single number to put there: hovering a multi-series chart now shows only the date. The legend directly below already updates every line's own figure for that exact day — that is where a per-series number belongs — and the fixed portfolio total already lives in Movement, labelled with the date range it covers. Single-series charts (a position's own history) are unchanged.
- No backend involved; frontend build and typecheck clean.
- 2026-08-29A third way to read the chart: euro change, not just percent
- Asked for directly: "gostava de poder ver tambem quanto é que um determinado holding cresceu em valor, e nao so em percentagem" — a small position that jumps 30% draws exactly as dramatic a line as a large one that moved 30%, even though the small one barely mattered to the total.
- Third Measure alongside Value/Growth: Change (€) — each series rebased to its own first day, as a euro difference rather than a ratio. Purely a frontend reading of the same numbers already on screen, following the precedent Growth already set (ADR — the server never learns a third shape of these figures either).
- Verified with two holdings on purpose sized to make the point: a large, steady position (+0.8%, +8,70 €) against a small, volatile one (−51.4%, −10,27 €). On Growth, the small one's line dominates the plot and the steady one reads as a flat non-event. On Change (€), both lines sit on a comparable scale and the reading matches what actually happened to the money.
- Same non-stacked, zero-anchored axis treatment as Growth (deltas can be negative and summing them across holdings is not a meaningful total), and the legend/readout/aria-label all gained a third branch alongside the existing value/growth ones.
- No backend change at all -- 661 backend tests unaffected. Frontend build and typecheck clean.
- 2026-08-29The CSV importer handles real files, not just clean ones
- Reported live: importing a real export ("2024-05-13,105.07,Hamburg, XS2800678224") gave "Added 0" and 582 unreadable lines. parse_bond_price_csv (ADR-037) was built for a strict two-field line and had no idea what to do with the extra columns a genuine broker export always carries.
- Rewritten around fields, not a single 2-way split: every field on the line is checked as a date candidate, and its price is found among whichever other fields parse as one — an exchange name or a repeated ISIN simply matches neither and is left alone.
- The harder part was making that coexist with the decimal-comma fix from the same ADR ("41,25" is one field split into two, not two fields). A reconstruction pass only glues two adjacent fields back together when both are bare digit groups — never when the first already contains a decimal point, since a field like "105.07" is already a complete number, and gluing a trailing column onto it ("105.07,50") would misread a real price plus an unrelated figure as one garbled number. Both shapes are now pinned by tests, including the two together in one line ("2024-05-13,105,07,Hamburg").
- Verified against 30 lines shaped exactly like the file that failed: 30 imported, 0 unreadable, correct euro values throughout.
- 661 tests pass (up from 658).
- 2026-08-29Fix a real 500: Tradegate's empty quote crashed the chart
- Found live, not in review: adding an ISIN and then opening the chart threw "Something went wrong on the server." Tradegate answers an ISIN it recognises but has no current trade for with "last": "", not null — present, but empty. The adapter's if last is None check let that empty string reach Decimal("") unguarded, raising an unhandled InvalidOperation that surfaced as an unhandled 500 all the way up through price_history.
- Fixed in infra/marketdata/tradegate_client.py: the presence check is now a falsiness check (if not last), and the Decimal() conversion itself is wrapped to turn any unparseable response into a clean MarketDataError rather than trusting a third-party payload's shape. daily_closes already caught MarketDataError and degrades to an empty series, so the fix is fully absorbed at the one adapter that produced it — nothing above it needed to change.
- New tests/test_tradegate_client.py — the first adapter-level test in this codebase; every other market-data client is exercised only through the service-level fakes. Justified here because a scripted response was what it took to reproduce this exact shape of failure. httpx.AsyncClient.get is patched directly rather than adding a mocking dependency this one regression doesn't otherwise warrant.
- 658 tests pass (up from 653).
- 2026-08-29Add a missing ISIN without deleting the bond
- Caught while testing: "No ISIN on file for this bond, so there is nothing to chart" had no way out except deleting the position and re-adding it — losing any manual price history already on file. The owner asked directly: "se calhar devia poder adicionar ISIN não?"
- New StockService.set_isin + POST /positions/{id}/isin — the one field a position can be edited for outside "remove and re-add" (that design choice stands for everything else), since the ISIN is the only thing an automated price feed needs. Same ISO 6166 check-digit validation as adding a bond with one from the start.
- The form for it replaces the "no ISIN" message on the spot rather than sending anyone back to "Add a position" — the fix sits next to the problem it fixes. Submitting it updates the position in place and immediately re-fetches the chart, so the screen goes straight from "nothing to chart" to "only one day recorded so far" without a reload.
- 653 tests pass (up from 649).
- 2026-08-29Import a whole file of bond price history
- Asked for directly: "tens de criar a opçao de um poder fazer upload de carregar cotacoes historicas." One day at a time through ADR-036's form is 250 round trips for a year of history; this is the bulk door onto the same table.
- parse_bond_price_csv (services/stocks.py) is deliberately tolerant: comma, semicolon or tab; either decimal separator; optional header; a trailing % or not; date-first or price-first, decided per line. The data is whatever the owner could actually get — a broker export, or a page saved from their own browser (a real browser sidesteps every wall ADR-036 hit) — never a format this app controls.
- Caught by the second test written for it, not by inspection: a bare comma cannot be trusted as a delimiter. "41,25" is one European-notation number; naively splitting on every comma silently truncated it to "41" — a wrong price with no error. Fixed by trying each comma position in turn and keeping the first split whose two sides actually read as a date and a percentage.
- Every unparseable line is reported back with its line number, never swallowed into a smaller count with no explanation. A day already on file is skipped and counted, not refused — unlike the single-day form, since a bulk import is expected to overlap with today's live-fetched quote, and refusing the whole file over that one expected collision would defeat the point of uploading one.
- POST /positions/{id}/bond-prices/import, multipart, mirroring the existing statement-upload convention. Frontend: the single-day form and a new file uploader sit behind two links on a bond's own history card, reporting "Added N · M already on file, skipped · couldn't read: …" in one place.
- No new table — writes into the same price_bars rows record_bond_price already does, under source="manual"; the chart and Tradegate merge from ADR-036 work unchanged. 649 tests pass (up from 636). ADR-037 recorded.
- 2026-08-29Bonds get a real price: ISIN, live quote, dated backfill
- "Porque é que as bonds nao aparecem?" led to "tenho a certeza que há preço de uma bond ao dia. ajuda se te mandar o link?" — the link (Boerse Stuttgart) sits behind Cloudflare and wasn't the answer, but the investigation it triggered found one that works: Tradegate, live-quote-by-ISIN, free, keyless, no browser-computed header, no JS challenge. Confirmed via OpenFIGI that the linked bond is real and actively quoted — the gap was never coverage, only reachability. Portuguese OT bonds, checked directly: trade on MTS Portugal, an institutional venue with no public feed at all.
- The owner chose the semantics: quantity is nominal held in euro; price is percent of face value. value = quantity × price ÷ 100 everywhere a bond is priced — deliberately not the price × quantity × fx_rate every other asset type uses, which would overstate a bond's value ~100x if applied to a percentage. This reinterprets what a manually-typed bond price meant before today (previously undocumented, never exercised with real data).
- A live quote is a percentage, stored as one, not forced into EUR cents: a quote like 41.252% carries precision that matters at real position sizes, and rounding it into minor units the way every other unit price is stored would lose a decimal a careful person would notice. "PCT" reuses the existing free-form price_currency column — no schema change — and exponent_for already gives an unrecognised code 2 decimal places, exactly what a percentage needs. The portfolio total is computed from the full-precision Decimal before the one rounding at the end, so this is a display choice only; api/schemas.py renders "41.25% of par" rather than the literally-true but unread "41.25 PCT".
- positions.isin (migration 1801ea70fc58), validated by its own ISO 6166 check digit (is_valid_isin in domain/models.py) before it's ever looked up — a typo is caught here, not surfaced later as "no data" or, worse, silently pricing a different, coincidentally-valid bond.
- TradegateBondPrices serves two ports, the way CoinGecko/Frankfurter already do (ADR-035): BondPriceSource for "Refresh value", and PriceHistorySource for the chart — the second by wrapping today's live quote as a one-day series, which is what lets a bond's chart build itself forward through the existing price_bars cache with zero new machinery.
- The one place with no automated answer at any price: history before today. New StockService.record_bond_price + POST /positions/{id}/bond-prices lets a person fill in a day from their own statement, written into the same price_bars table under source="manual" and merged with Tradegate's own fetches by date. Refused outright on a day already on file — nothing in one number can tell a correction from a duplicate slip, and a cache that already promises "a settled day never changes" is not the place to guess which this is.
- Frontend: an ISIN field on bond positions ("lets FinAid price it and chart it itself"), the quantity field relabelled "Nominal held (EUR)" for bonds only, the manual-price field relabelled "% of face value" with no currency picker, and a collapsed "Add a price you know" form on a bond's own history card. Caught on screen: the position row read "1000 units" for a bond (implies 1000 bonds, not €1000 nominal) — now "1000 EUR nominal"; and the chart's resolved-symbol sentence, written for a looked-up ticker ("VWCE.DE, the listing Yahoo recognised"), read as if Tradegate had done symbol resolution when the ISIN was simply typed in — bonds get their own sentence now.
- Verified against the real Tradegate endpoint, not a fake: a German Bund, 1000 EUR nominal, live quote 41.252% → 412.52 EUR exactly. 636 tests pass (up from 624). ADR-036 recorded.
- Known, unrelated, not touched: test_history.py::test_limit_is_respected_and_capped fails against a hard-coded entry count that drifts as this changelog grows — confirmed failing before this session's changes too.
- 2026-08-29ADR-036Bonds get a real price: an ISIN, a live quote by percentage, and a way to fill in yesterday
- "Porque é que as bonds nao aparecem?" surfaced two problems, one shallow and one real. The shallow one: a bond has no daily-history source (ADR-035), so the chart correctly named it as excluded. The real one: the manual price the owner types in is used by "Refresh value" — so the page carried two figures for "what is my portfolio worth", a hand's width apart, with nothing saying why one counted bonds and the other didn't. Fixing the silence (ADR-035's second follow-up) was a start; the owner then pushed further: "tenho a certeza que há preço de uma bond ao dia" and asked for the actual source.
- 2026-08-29ADR-037A bulk door onto the same table: importing bond price history
- ADR-036 gave a bond a live quote and a way to fill in one day by hand, since no source publishes bond history to an automated client. One day at a time does not scale: a year of history through that form is 250 round trips through a date picker. The owner asked directly for the option to upload historical quotes.
- 2026-08-28Two totals on one page stop contradicting each other
- Asked: "porque é que as bonds nao aparecem?" They have no daily price source (ADR-034 — that is why you type the price yourself), so _history_source_for returns None for a bond and the chart names it under "Not included". But the question exposed something worse: the manual price is used by "Refresh value", so the page carried two figures for "what is my portfolio worth" — one including bonds, one not — a hand's width apart and with nothing saying why.
- The valuation card now names the manually-priced holdings it counts and says the chart leaves them out. Also switched that card from the API's locale-neutral *_display strings to formatSpend, since it sat directly above a chart already using them: 16909.75 EUR over 20 759,33 €.
- Not fixed, and the owner is right to push on it: bond prices do exist daily. What is missing is a reachable source — Yahoo does not cover individual government debt, Euronext encrypts its quote endpoint against robots, Boerse Frankfurt wants a browser-computed security header, Stooq serves a JavaScript challenge — and, more fundamentally, an ISIN. A position stores "OT 2030" as free text, and every bond price source is keyed by ISIN. Both are prerequisites before any of this can be automated.
- 2026-08-28Two controls, six readings of the portfolio chart
- Asked for: the chart split by asset type, growth by asset type, growth by ticker — and the owner spotted the shape themselves, "na verdade sao duas dimensoes, por isso so deviam ser necessários dois botoes certo?" They are. What it measures (Value / Growth) and how it is split (Total / By type / By holding) are independent, so they are two pill groups and all six combinations mean something.
- Only the split reaches the server: group_by on GET /api/stocks/value/daily returns series sharing one date grid. A test asserts the slices sum back to the ungrouped total, day for day — a breakdown that does not reconstitute the whole is not a breakdown. Growth is computed in the chart, because it is the same series rebased to its own first day; a measure=growth parameter would imply the server holds figures it does not.
- The axis changes with the question, on purpose. A single line keeps the tight scale; a breakdown is a composition and stacks from zero, because parts of a whole cannot be read against a floating floor; growth always includes 0% and gets a rule there. Each case says which it is above the plot.
- The legend doubles as the readout — with several lines, a tooltip on one of them answers about the wrong one as often as the right one, so hovering moves every legend figure to that day at once. The headline stays the portfolio total in all six readings: a split is a partition, and the figure someone came for should not vanish because they pressed "By type".
- Six series colours as CSS custom properties, hues rather than shades, with the accent green first so an ungrouped chart and a grouped one still look like the same picture. Bands are separated by a hairline in the card's own background rather than a darker edge that would read as a seventh colour.
- 624 tests pass (up from 621). One self-inflicted scare: a slice-and-splice edit whose end marker sat before its start marker duplicated 260 lines of the page; the typechecker caught it immediately with a wall of "duplicate function implementation".
- 2026-08-28High, low and opening, drawn on the chart
- Asked for: three horizontal rules — the maximum, the minimum and the opening — each with a percentage against the current value, "mas isto discreto". Added to PriceChart, so both the portfolio chart and every holding's own chart get them from one renderer.
- Discreet means hairline, dashed, --border-coloured and drawn underneath the data: they are a grid to read the line against, not three more lines competing with it. The percentages are from each level to where the line ends now, so "open" restates the period's return and "high"/"low" say how far the current value sits from each edge of the window.
- Two placement problems, both only visible on screen. The high sits on the ceiling, where a label above its own rule came out sliced in half by the edge of the drawing — that one now hangs below its rule instead. And on a rising line the high label lands exactly where the data is, with no edge of the plot reliably empty, so the labels carry a paint-order: stroke halo in the card's own colour rather than being moved somewhere that only works for some shapes. Verified in both light and dark.
- Labels are nudged apart when their levels nearly coincide — a line that only fell has its opening and its high on the same value, and a flat one puts all three there. The rules stay where they belong; only the text moves.
- The chart is role="img", so its inner text is not announced: the three levels are spelled into the aria-label too, or they would exist for sighted readers only.
- 2026-08-28The portfolio chart runs on real closes
- Asked for: feed the portfolio chart from the daily closes, "para ja assume que eu tenho todas as minhas posicoes desde sempre". So the line is a counterfactual — today's quantities priced at every past close — and the card says exactly that, in the plainest sentence on the page: buying more of something moves the whole line, not just its right-hand end.
- StockService.portfolio_history + GET /api/stocks/value/daily. Three rules keep the total honest: it starts only where every included holding has a price (a total that grows as assets join it reads as gains); a shut exchange carries its last close forward rather than counting as zero (crypto trades on Sunday, a stock does not — treating the gap as nothing would draw a crash every weekend); and a holding with no daily history is named on screen, never quietly dropped.
- It also says which holding decides where the line starts — but only when one genuinely truncates it. The first version blamed Apple, listed since 1980, for a Saturday: holdings on different exchanges disagree about which day a window opens on, so the claim is only made past a week's difference. Caught on screen at the 3M range, and now pinned by a test.
- The old snapshot chart is demoted to a disclosure under the new one, not deleted: until movements exist, those snapshots are the only trace of what was actually held, each computed with the quantity that stood at the time.
- Next step, deliberately left open: recording movements (a dated buy/sell per position) so the chart reads quantity as of each day. That is what will make the two charts agree, and what turns this line from a counterfactual into a record.
- 621 tests pass (up from 616). PriceChart and Movement now take one ChartPoint shape so the holding and portfolio charts are the same renderer at two scopes rather than two that drift apart.
- 2026-08-28A holding opens its own chart, in place
- The screen half of the daily closes above. Each declared position is now a control that opens its own year of value underneath it (PositionHistoryCard in web/app/stocks/page.tsx), with 1M/3M/1Y/2Y pills — the same answer-where-you-are move the dashboard made, for the same reason.
- This axis does not start at zero, and says so. The portfolio chart right above it is zero-based on purpose (ADR-034: three refreshes must not look like a crash); zero-basing a year of daily closes would flatten every real move into a straight line. Different question, different axis — so the window's low and high are printed above the plot and the scale is stated in words.
- Hover reads out the day under the pointer (value, date, unit price) through one handler over the plot rather than a hit target per day — a year is 250 of them. Each card also states what it is not: today's units against past prices, converted at the rate that stood on the day, and the listing actually charted when it differs from the ticker.
- Three defects the screen showed that nothing else would have: money arriving as the API's locale-neutral 2701.42 EUR beside 709,64 €; a unit price reading 247.649993896484 USD, which is Yahoo's 32-bit float for 247.65 (rounded for display only — the stored value stays exactly what the provider said); and .value-chart drawing itself in a 600px column in the middle of a 900px card, because a fixed height made the viewBox scale to fit rather than fill. The last one was already true of the portfolio chart and is now fixed for both.
- CoinGecko's free tier serves at most a year of daily history and answers a longer window with a 401, so the adapter clamps to a year rather than raising: a 2Y window on a coin draws the year that exists, and the axis says where it starts. Its 429 now explains itself instead of showing a status code.
- 2026-08-28StockAId fetches real daily closes, once and for good
- Asked for: history for the tickers already added, "só precisa de ir buscar uma vez porque esses nunca vao mudar". The portfolio chart from ADR-034 only ever knew the moments someone pressed "Refresh value"; daily closes are the actual shape of an asset.
- Finnhub, the incumbent price source, answers /stock/candle with a 403 on its free tier — so this is a new port, PriceHistorySource, not a method on StockPriceSource (a port whose only adapter could never implement it is not a port). Yahoo's keyless chart endpoint fills it (infra/marketdata/yahoo_client.py), CoinGecko's market_chart covers crypto, and Frankfurter's time series covers FX — because an exchange rate is a price bar: one dollar cost 0.8654 euro on the 10th, same shape as a share close, so it caches through the same table and the same code.
- Two new tables, the first in this schema with no owner_id: a closing price is a fact about a market, not about a person. price_bars holds the closes, unique on (source, symbol, day) so "fetch once, ever" is enforced by the schema. price_series_coverage holds the window that has been asked for, which is a different fact — a stock listed in 2021 has a permanently empty stretch in front of it, and a bars-only cache would re-request that emptiness forever. Migration bfb04c364e77, which also adds positions.history_symbol.
- StockService.price_history + GET /api/stocks/positions/{id}/history. Past closes are converted at the rate that stood on the day, carried forward over weekends and never backwards; a close older than any published rate is dropped rather than converted at a rate that did not exist. Today's quantity is applied to every past day — this app has no trade history — so the series means "what today's holding would have been worth", and quantity travels out to the screen so it can say so.
- A bare ticker a provider does not recognise ("VWCE") is looked up once and the resolved listing ("VWCE.DE") is reported, never silently substituted — charting a different listing without saying so would make the feature worse than not having it.
- 616 tests pass (up from 608), including one that counts HTTP calls: the second request for a window must reach the provider zero times, not fewer. That test is what caught the empty-stretch bug above. Verified end to end against the real providers — four positions over six months, four calls on the first visit, zero on the second. ADR-035 recorded.
- Known, untouched: alembic check reports a pre-existing drift (ix_parsing_hints_owner_id exists in the model but in no migration). Left alone rather than folded into an unrelated migration.
- 2026-08-28ADR-035Daily closes are cached forever; a portfolio chart is not a substitute for one
- ADR-034's third follow-up drew the portfolio's value over time from position_valuations — the moments someone happened to press "Refresh value." That chart is honest about what it is, but it is not the shape of an asset: three presses make a three-point line, a fortnight away makes a straight one, and neither says anything about how the holding actually moved. The owner asked for the real thing: "gostava que o agente fosse buscar os dados historicos para o tickers adicionados, pelo menos o closing value ou algo semelhante. Só precisa de ir buscar uma vez porque esses nunca vao mudar."
- 2026-08-27The sort moved onto the table it sorts
- Asked for: the sort as "dois discos no canto superior direito da tabela". It was a single seven-option <select> buried in the filter card above the list — the control that reorders the table sat a screen away from it, and reversing an order meant reopening a menu and reading seven labels to find the one that was the same as the current one backwards.
- Now two lozenges on the list card's own header row, right-aligned opposite the existing "Detail" control (SortPills in web/app/transactions/page.tsx, .table-tools / .sort-pills / .pill-select / .pill-toggle in globals.css): a field pill (Date, Amount, Name, Bank) and a direction pill that reads as the order it is ("↓ Newest first"), not as what pressing it would do — the person is reading a table and needs to know how it is stacked; the action lives in the accessible name. Four fields x two directions covers what seven options did, and reversing is one tap. The wire format stays one field_direction string, so nothing below the component knows the control was split.
- Which required date_asc to exist at all: the ordering map named every key except that one, so "oldest first" was unreachable and, worse, would have fallen through to the default and shown the exact opposite. Added to the repository's ordering map and to the route's Literal; pinned by test_oldest_first_is_a_real_ordering_and_not_the_default.
- The field pill is a native <select> laid transparently over visible text rather than a rebuilt popover, so a phone opens its own picker and a keyboard gets arrow keys for free. First attempt positioned the select absolutely and relied on it for width, which collapsed the lozenge to a sliver reading "/⌄" — caught on screen, not by the typechecker.
- 608 tests pass (up from 607). Verified in Chromium at 390px and 1200px: the pair sits inside the list card's top-right corner at both widths, flipping the direction genuinely reverses the rows, and changing the field keeps the direction it was already in.
- 2026-08-27Standing charges, per year or per month
- Asked for directly: "nas despesas recorrentes gostava de poder ver por ano mas tambem por mes". Both readings answer different questions — the yearly figure is what makes somebody cancel a subscription, the monthly one is what they check against a salary — so this is a unit switch, not a filter: the list, its order and which charges count toward the total are identical either way.
- RecurringItem gained monthly_minor (services/transactions.py), derived from the yearly figure rather than from the charge, so a quarterly bill reads as its share of a month instead of as something that leaves monthly. RecurringSummary.monthly_minor is now summed from the rows' own monthly figures instead of dividing the yearly total by twelve, so the headline is the sum of the column beneath it down to the cent in both units. RecurringItemOut exposes monthly_minor / monthly_display; types regenerated.
- New shared web/lib/recurringBasis.ts (the Basis type, a cost() reader, and a localStorage-backed useRecurringBasis hook) and web/app/components/BasisSwitch.tsx — real radios under the visible pills, reusing the .density control's styling under a shared .segmented name. The preference is shared between /recurring and the dashboard's recurring panel, since finding one in years and the other in months would read as a bug.
- The dashboard's recurring stat card now quotes the same costed figure as the panel it opens (fetched on arrival rather than on first click), replacing an average-times-twelve estimate that disagreed with its own panel. While there, /recurring and that panel switched from the API's locale-neutral *_display strings to formatSpend, so "13 556,00 €" no longer becomes "-13556.00 EUR" on the way between two screens showing the same number.
- 607 tests pass (up from 606): the recurring-total test now asserts the monthly headline is summed rather than divided, and a new test pins that a monthly reading spreads a quarterly bill rather than repeating it. Verified in Chromium against a throwaway SQLite database: switching units leaves the row count and order untouched, every row is its own yearly figure over twelve, the headline equals the sum of the fixed rows shown, and the choice survives a reload and carries to the dashboard.
- 2026-08-27StockAId: a chart of portfolio value over time
- The owner asked for a way to see how their declared portfolio has moved, not just its value "just now." position_valuations (ADR-034) was already the right shape for this — an append-only row per position per record_portfolio_value call — so this is a read, not a new aggregate: every row from one call shares an identical computed_at (StockService.record computes it once, outside the per-position loop), so summing value_minor grouped by that exact timestamp reconstructs each past call's total without a separate "portfolio total" table or any migration.
- New SqlPositionValuationRepository.list_totals_for_owner (SQL GROUP BY computed_at, portable — verified on SQLite and Postgres 16), StockService.value_history, GET /api/stocks/value/history (returns computed aggregates only, consistent with CLAUDE.md #6, never the per-position rows behind them), and PortfolioValueHistoryOut in api/schemas.py.
- web/app/stocks/page.tsx gained a "Value over time" card: an inline SVG line/area chart, fetched on load (unlike "Refresh value," this is a plain read — no external pricing API call, nothing new persisted) and again after each refresh. Zero-based y-axis on purpose, so a small refresh-to-refresh move never reads as a bigger swing than it is. Fewer than two points shows a one-line hint instead of a degenerate chart.
- 605 tests pass (up from 602), including new TestValuingThePortfolio::test_value_history_* in test_stocks.py and two HTTP-level tests in test_api.py. Verified by driving the real page in Chromium against a throwaway SQLite database (not the same one this suite uses) in both light and dark mode: the line, area fill and end marker track a genuine three-point rising trend correctly, and per-point hover tooltips read date and value.
- 2026-08-27A transaction row you can read, and a review you can do in one tap
- Reported as "muito cluttered": a row rendered up to seven pieces of metadata as equal-weight chips — date, bank, category, recurring / semi-recurring, "check", "not on a statement", "checked by you" — so nothing stood out, including the one chip actually asking for something. The row now has three deliberate weights: description and amount, then quiet meta text (date, category), then a chip reserved for states that want an action. Recurrence became a glyph instead (RecurringIcon / SemiRecurringIcon, solid vs dashed — the same distinction the solid vs outlined chips drew), because it is true of the merchant forever and needs no action.
- New Detail control on the list, Compact (default) / Everything, as a real radiogroup, remembered in localStorage. Bank name, the two "matched with…" lines, not on a statement and checked by you show only in Everything. Compact rows are pinned to exactly two lines (.txn-row[data-density="compact"], meta flex-wrap: nowrap, category ellipsised): a long category name wrapping to a third line gives the list ragged heights, which is the scanning problem compact exists to fix.
- POST /api/transactions/{id}/reviewed — tick a row off the review queue without opening it, or put it back. Deliberately not a field on TransactionPatch: update() sets confidence = 1.0 and classified_by = "user" because a correction really is a human classification, but agreeing with a guess is not a reclassification, and overwriting the original confidence is exactly what would make an undo impossible. TransactionService.set_reviewed therefore moves only user_verified / needs_review, which makes the undo an exact restore rather than a reconstruction — asserted by test_api.py::TestTransactions::test_acknowledging_a_row_can_be_undone_exactly.
- The tick button is also the undo (aria-pressed), since a one-tap action needs a one-tap way back and the row is where the person is already looking. A row ticked off stays put and dims rather than vanishing — disappearing under the finger mid-pass is how rows get missed — and is gone on the next load. No "check" chip any more: the button only ever appears on a row that needs checking, so a chip saying so too was the same fact twice, and it wrapped.
- Fixed alongside, both found by reading this screen: the transactions list showed "No transactions match" underneath a network error, giving one screen two contradictory explanations (it now returns early on error, as /overview already did); and each row's aria-label was ` Edit ${description} `, which replaces rather than supplements the row's contents — amount, date, category and every state were invisible to a screen reader. It now names them.
- Verified by driving the real screens in Chromium at 390px and 1280px against a throwaway database, not only by the suite: the tick/undo round-trip leaves every field byte-identical, and compact row heights measure uniform. 594 backend tests pass.
- 2026-08-27The dashboard answers in place instead of sending you away
- Reported: "assim que carrego num dos quadrados sou logo mandado para as transactions… manda-me para lá demasiado cedo". Correct, and an inconsistency of ours: category rows had already been pointed at analysis rather than the ledger, but the four stat cards still jumped straight to it. A total is a question, and a filtered list of rows is not its answer.
- The cards are now a tablist, not links. Pressing one keeps you on the page and rebuilds the panel below it: spent → money out by month plus "Where it went"; received → money in plus "Where it came from"; net → the zero-line chart; recurring → the yearly commitment and the costliest standing charges. Real role="tab"/aria-selected/aria-controls, since that is what the control genuinely is.
- The ledger is still one tap away, but at the bottom of what you came to understand rather than as the first thing a figure does.
- MonthChart now draws one direction at a time. Both bars at once made it a comparison of income against spending — a third question, and not the one either card poses; /net answers that one properly with a zero line.
- The net panel deliberately ignores the period pills and always shows six calendar months. "Am I usually keeping anything?" is not a question about the window you happen to have selected, and the first build of this read overview.by_month — which, with the default "This month", left the panel empty. Caught by driving it, not by the types.
- /net, /recurring and /category/[slug] stay as the deeper view behind each panel; the dashboard now carries the compact version of each.
- Verified in Chromium at 390px: four tabs, exactly one selected at a time, every switch stays on /overview, and each panel renders its own heading. 606 backend tests pass — this change is frontend only.
- 2026-08-27Nothing on the dashboard that does not open
- Last of the four stat cards to be inert. "Left over" was one window's arithmetic: it answers "did I keep anything this month" and stops, where whether you usually keep anything is the question that decides if something has to change.
- New GET /api/net-trend?months= and TransactionService.net_trend, reusing totals_by_period and the _month_keys zero-fill introduced for category_trend. Every card on /overview now opens what it is made of.
- Empty months are drawn but never judged. The chart needs every bucket or the months slide along the axis and misreport the shape of time; the summary needs the opposite, since counting a month with no movements as "in surplus" would flatter a ledger that simply has a gap in it. Asserted by test_the_net_trend_draws_empty_months_but_does_not_judge_them.
- web/app/net/page.tsx draws the months either side of a zero line rather than as bars from a floor: which side of it a month falls on is what should read before any number does. Figures here are signed, unlike spending elsewhere in the app — a deficit rendered through formatSpend came out as a positive number in red, leaving colour alone to carry the difference between a month you kept money and one you did not.
- Verified in Chromium at 390px: all four stat cards are links, the chart puts four months above the line and two below, and the range pills refetch (12 columns for a 12-month window). 606 backend tests pass.
- 2026-08-27Where the money leaks: standing charges you can act on
- The dashboard has carried "Recurring, approx. per month" since the first version and it went nowhere, which made it the one figure on the page nobody could act on. The owner called it one of the most important: someone trying to spend less starts with the charges that leave without a decision being made each time.
- New GET /api/recurring and TransactionService.recurring, over the recurring_groups aggregate that already existed. Costed from each charge's own cadence, not multiplied by twelve: the gaps between the charges on file give an average interval, so a quarterly insurance bill is billed four times a year rather than twelve. Ordered by yearly cost, because "what is worth cancelling" is a question about size.
- The headline is the sum of the rows beneath it. Built from the same per-charge yearly figures rather than from average x 12, since a total that disagrees with its own list is the fastest way to make a screen untrustworthy. Asserted by test_the_recurring_total_is_the_sum_of_the_rows_it_shows.
- A charge silent for more than twice its own usual gap is marked looks_inactive and moved behind a disclosure — not asserted as cancelled, since a statement may simply be missing, but the difference between money still going out and money that stopped.
- New web/app/recurring/page.tsx, split into fixed commitments (what the total counts) and things that repeat with a varying amount (groceries, fuel — real spending, but not a commitment). The yearly figure leads: a subscription at "13,99 € a month" is easy to wave away where "196,40 € a year" is not.
- Fixed while testing this on a machine west of Greenwich: formatDate parsed "2026-08-01" as midnight UTC and rendered it in the viewer's zone, so every booking date displayed a day early anywhere behind UTC. A booked_on is a calendar day, not an instant, and is now built with the local constructor. Invisible from Portugal, wrong everywhere west of it.
- Verified in Chromium at 390px: the card opens /recurring, the headline equals the sum of the fixed rows, a quarterly charge costs ~4x not 12x, and the quiet one sits behind its disclosure. 600 backend tests pass.
- 2026-08-27A category over time: the level the app was missing
- Reported as "não gosto nada do overview… zero intuitivo", with a concrete example: how do you see whether a category is worse than last month? You could not. Every figure on the dashboard is one window, and a single window has no direction; clicking a category jumped straight to the ledger, which answers "which rows" when the question was "why".
- New GET /api/categories/{slug}/trend?months= and TransactionService.category_trend: one category over calendar months, plus each subcategory's share and which way it moved. No new SQL and no migration — totals_by_period already accepts a TransactionFilter with categories, so this composes aggregates that existed and were already proven on both engines.
- Calendar months, decided with the owner. That does not contradict the rule against treating a statement period as a calendar month (#4): that rule governs whether a document is complete, which is arithmetic on its own balances. This is only how spending is bucketed for a chart.
- Gaps are filled with zeroes (_month_keys). A GROUP BY only returns months that have rows, so a month you spent nothing in simply vanished — and a chart missing a month from the middle misreports the shape of time.
- A percentage is never invented against a missing baseline. No previous month means change_vs_previous_pct is null, not a division against zero; months_with_data travels alongside so a caller can decline to draw an arrow from too little. The screen stays quiet below three months and says why. Asserted by test_a_trend_never_invents_a_percentage_against_a_missing_month.
- New web/app/category/[slug]/page.tsx: range pills (3/6/12 months), the month chart with the current month at full strength, both comparisons shown with the euros they came from (a bare "▲34%" is unreadable), a "What moved" breakdown, and the transactions as an exit at the bottom rather than as the only thing a click can do.
- Overview rewired to the principle the owner asked for — nothing that is not clickable: category rows now open the analysis instead of the ledger; the "Where it came from" rows open that transaction in EditDialog in place, where before naming a transaction and opening a filtered list meant tapping "Ordenado" did not take you to Ordenado; and "Spent" joins "Received" as an openable card, both carrying a chevron so a card you can open looks different from one you can only read.
- "Left over" and "Recurring, approx." are deliberately still inert: their screens (net over time, and the subscriptions behind the estimate — recurring_groups already returns everything one would need) do not exist yet, and a door that opens onto nothing teaches people to stop trying.
- Verified in Chromium at 390px: a category row lands on /category/food, the range pills refetch (12 buckets vs 3), and the inflow row named "Ordenado" opens a dialog titled "Ordenado". 597 backend tests pass.
- 2026-08-27Four consistency fixes across the web app
- Spending bars are no longer the colour of good news. --accent and --positive hold the same value, so .bar-fill — the dashboard's "Where it went" bars — was painting expenses in the hue the "Received" card two cards above uses for money arriving. Expense bars now take --negative, matching the amounts printed beside them; "Where it came from" opts back in with .bar-fill.inflow, so the two blocks are told apart by colour and not only by their heading.
- The four transaction filters use the app's own checkbox. They were four near-identical inline-styled labels wrapping a bare native input (~13px), while .checkbox — 20px, themed accent-color, commented as being big enough to hit on a phone — was already used by EditDialog, TeachBox, Receipts and Delete. Now one .filter-checks stack of shared .checkbox rows.
- New ConfirmDialog, replacing the last two window.confirm() calls (EditDialog, stocks). The native dialog ignores the theme, carries the system's typography, and on an installed PWA reads as the OS interrupting the app — one of the two was appearing on top of this very modal. It also cannot say more than a sentence, which was the real cost: "Remove this movement?" never mentioned that the row counts in every total until it goes, or that a later statement brings the bank's own record back.
- EditDialog is grouped and its actions are pinned. Up to nine fields ran as one undifferentiated column; they now sit under three headings — what this is / how it relates to other transactions / how often it repeats — and .modal-actions.sticky keeps Save reachable without scrolling past everything to reach it. "Remove this movement" gets its own rule above them, rather than sitting flush against Save.
- Verified in Chromium at 390px: filter checkboxes measure 20×20, the three section headings render, Save stays on screen both at the top of the form and scrolled to the end, the custom confirmation opens with zero native confirm() calls fired, and the two bar blocks compute to #b3261e and #1f6f5c respectively. 594 backend tests pass.
- 2026-08-27The way out of ExpenseAId, on purpose rather than by accident
- .nav a (specificity 0,1,1) was beating both of .nav-brand's own display rules (0,1,0), so neither the display: none nor the display: block in the ≥768px block ever applied and the brand computed to display: flex at every width. It was visible, but by accident and wearing the wrong clothes: a sixth, icon-less item in a phone bar whose own comment says "Five is what fits across a phone", and the wordmark at .nav a's 0.9rem on the sidebar instead of its intended 1.05rem.
- Kept — the owner wants the wordmark on the left as the way through to StockAId — but made deliberate. Every rule is now written .nav a.nav-brand (0,2,1) so it wins on specificity rather than on luck, and the base rule sits above the media query, since matched specificity is settled by order and a base rule written after it would beat the sidebar's own overrides. On a phone the brand is a fixed 58px with a hairline separating it from the tabs, so the five keep the room they were sized for (65–72px each) instead of each surrendering a sixth of it; on the sidebar it is the masthead it was always meant to be.
- New AgentsIcon — four panes, not a letterform of icon.svg. Drawn in strokes at 20px the mark's F-plus-dot reads as the letters "Fi", so beside a label already saying "FinAid" the word appeared to start twice. The label carries the brand; the glyph says what tapping it does.
- The brand is also data-active on / and /stocks. StockAId matches no tab, so before this it was a screen with nothing at all highlighted in the navigation.
- Verified in Chromium at both widths: the brand measures 58px on a phone and the full 185px sidebar width on a laptop, and Brand → Stocks leaves "FinAid" as the highlighted item.
- 2026-08-26StockAId resolves a name to a symbol, for stocks, ETFs and crypto
- symbol_or_identifier for a stock, ETF or crypto position now goes through the model before being saved: "Mastercard" resolves to "MA" (Finnhub's vocabulary), "Bitcoin" resolves to "bitcoin" (CoinGecko's own id, not the "BTC" ticker) — instead of failing at valuation time with "no quote for 'Mastercard'." Deliberately model-based rather than a provider symbol-search endpoint, to avoid a second dependency on the same vendors already backing StockPriceSource/CryptoPriceSource. Bonds are exempt — a free-text name is all there is.
- StockService._resolve_symbol (called via _resolve_ticker for stocks/ETFs, _resolve_coingecko_id for crypto) mirrors RuleService.interpret's shape (system prompt + one LLMClient.extract() call + a small JSON schema) but diverges in outcome: below MIN_SYMBOL_CONFIDENCE (0.7) or an empty result, StockService.add raises ValidationError directly rather than returning an "unclear" result for the caller to decide about — a saved wrong symbol silently prices the wrong asset, or nothing at all, which is worse than asking again. Wired to extraction_llm (claude-haiku-4-5), the same model RuleService/ClassificationService already use for routine classification.
- add (and the PositionWriter port behind it) is now async; both existing call sites already ran inside async functions, so this was a mechanical await at each.
- ADR-034 amended with two same-day follow-ups (stocks/ETFs, then crypto). 591 tests pass.
- 2026-08-26A FinAid landing page, and a UI for StockAId
- FinAid gained an actual umbrella page: / used to be the expenses dashboard directly (no landing page existed at all); it now shows two tiles, "Expenses" and "Stocks," and the dashboard moved unchanged to /overview. Nav.tsx's "Overview" tab repoints there; the rest of the five-tab bottom nav is unchanged. nav-brand ("FinAid") is now a link home.
- New REST surface for StockAId, api/routes/stocks.py: GET/POST /api/stocks/positions, DELETE /api/stocks/positions/{id}, POST /api/stocks/value — mirrors api/routes/todos.py's add/list/delete shape exactly; StockService.update stays unexposed, matching the MCP surface's "add only, edit stays with a person" rule.
- New web/app/stocks/page.tsx: an add-position form (asset type, ticker/ coin-id/name, quantity, and a manual price only for bonds), a position list with remove, and a "Refresh value" button — deliberately not fetched on page load, since it calls real pricing APIs and persists a new valuation snapshot every time.
- Fixed while building this: deleting a position that already had a computed valuation violated the position_valuations foreign key and surfaced as a generic 503 "database unavailable" instead of succeeding — a real bug the UI reached that MCP-only usage had not. SqlPositionRepository.delete now deletes a position's valuation snapshots alongside it. Covered by a new test in both test_stocks.py and test_api.py.
- ADR-034 amended with a same-day follow-up section recording this. 582 tests pass on SQLite and Postgres 16.
- 2026-08-26StockAId: declare what you hold, see what it's worth
- New sub-agent agents/stocks/ (ADR-034), resolving the holdings/valuation question ADR-017 left open. Three tools: add_position (stocks, ETFs, crypto, or bonds), list_positions, and record_portfolio_value, which fetches current prices and returns the total in EUR plus a per-position breakdown.
- New aggregate, migration d51540a93a42: positions (mutable, owner-declared holdings) and position_valuations (append-only — every record_portfolio_value call writes a fresh snapshot rather than updating a single "current value"). quantity is Numeric, not the usual amount_minor: int, because it is an asset count, not currency.
- Three price routes: stocks/ETFs via Finnhub, crypto via CoinGecko (already EUR), a non-EUR stock quote converted through Frankfurter. All best-effort, not guaranteed real-time — free equity data outside a US/IEX exception is industry-wide delayed at least 15 minutes, accepted rather than paid around. Bonds have no automated source at all (no viable free/cheap corporate-bond API exists); the owner supplies a manual price, and a bond without one is reported but excluded from the total rather than failing the whole call.
- add_position and record_portfolio_value are the second deliberate widening of the ADR-016 "only add_todo writes" exception — both only ever append self-declared data, never mutate or delete existing corroborated history. tests/test_todos.py::TestNothingElseWrites now asserts the full three-tool set. MCP/chat only at first; a web UI followed the same day — see the entry above.
- 2026-08-26Categories can be shaped by each person
- Added a responsive Categories screen under More. A person can create a top-level category or subcategory, rename it, move a leaf between groups, hide it from new choices, restore it, or reset personal changes. The editor explains that hiding is reversible and never removes financial history.
- Seeded category rows and persisted slugs remain stable. Migration c6e4a1b9d2f7 adds category_preferences, which overlays label, parent and visibility per owner; one person's “Supermarket” does not rename another person's “Groceries”. User-created categories keep their owner-scoped base row and receive collision-resistant internal slugs that the UI never exposes.
- Hidden categories remain in GET /api/categories so old transactions keep readable labels and the management screen can restore them, while pickers, classification prompts and category-assignment rules exclude them. Hiding a group also hides its direct children. uncategorised cannot be hidden because it is the classifier's safe fallback.
- Dashboard rollups now apply the owner's effective parent after the SQL aggregate, so moving a subcategory immediately moves its total without rewriting transactions. ADR-033 records the data-preserving boundary.
- Verified with the 555-test backend suite, migration upgrade/downgrade/upgrade on SQLite, the generated OpenAPI contract, TypeScript checking, and a production Next.js build.
- 2026-08-26Pair a purchase with its refund, and see where money came from
- Overview gained a "Where it came from" tile, mirroring the existing "Where it went" categories: the period's largest individual inflows, each linking straight into Transactions filtered to money received. The "Received" stat card is now itself a link to that same filtered view — both reuse the only_inflows transaction filter added alongside them (TransactionFilter.only_inflows, amount_minor > 0, the mirror of the existing only_outflows).
- New is_refund_match/refund_pair_id on Transaction (migration e2b7f4a91c68) let a person explicitly pair a purchase with its refund. Deliberately not a reuse of is_internal_transfer: ADR-028 already treats a same-bank, opposite-amount coincidence as more likely a refund than a transfer, so conflating the two would make that evidence untrustworthy. TransactionService.refund_candidates/match_refund mirror the transfer equivalents; once paired, both legs disappear from every total the same way (TransactionFilter.include_refund_matches, default off, one more clause in SqlTransactionRepository._apply). EditDialog gained a second, mutually-exclusive checkbox for it; the transactions list gained a ↩ refund match chip and a Show refund matches toggle. See ADR-032.
- 549 tests pass on SQLite and Postgres 16 (up from 543), including new TestRefundMatchesAreExcludedByDefault/TestManualRefundMatching in tests/test_overview.py.
- 2026-08-26ADR-032A refund is a matched pair, not a transfer
- A purchase that gets fully refunded produces two real, correctly-imported transactions: an outflow when it was bought, an inflow when it was returned. Both are genuine spending and genuine income at the moment each happened, but showing both separately double-counts an event the person experienced as "I didn't actually keep spending that money." ADR-028 already considered and rejected folding this into is_internal_transfer: two coincidentally opposite amounts at the same bank are treated there as more likely a refund than an unannounced transfer, precisely because a refund is not money moved between the owner's own accounts — it is money returned by whoever was paid. Reusing the transfer flag for a refund would make that same-bank transfer evidence untrustworthy in the other direction.
- 2026-08-26ADR-033Category customisation is an owner-specific overlay
- Categories were structurally present and user-created categories could emerge from a learned rule, but a person had no direct way to maintain the taxonomy. Editing a seeded categories row would rename or move it for every owner. Changing a slug would be worse: transactions and learned rules persist that slug, so a cosmetic edit could orphan financial history. Physical deletion has the same problem for any category that has ever been used.
- 2026-08-26ADR-034StockAId: positions are declared, valuations are snapshots, and two more tools may write
- ADR-017 named StockAId as a sibling of ExpenseAId and BudgetAId but deliberately left it unbuilt: "holdings introduce a value that changes with no transaction and nothing to corroborate it. That deserves its own ADR rather than arriving as a side effect of 'adding crypto.'" The owner now wants v1: tell the agent what positions you hold — stocks, ETFs, crypto, bonds — and have it fetch current prices to show total portfolio value in EUR. Two things had to be decided before that could be built, not just the holdings/valuation question ADR-017 flagged.
- 2026-08-25Chat correction requests open a real rule preview, not a to-do
- Added the read-only preview_transaction_correction ExpenseAId tool for requests such as “consider every 940 euro payment to André as MBA.” It returns the person's instruction unchanged and tells the chat surface to present it for confirmation; it does not create or alter a financial row.
- The chat UI renders that tool call through the existing TeachBox, prefilled and automatically interpreted. The user sees the proposed rule, target category, examples, and affected transaction count, then explicitly presses Apply and remember. The existing feedback endpoint remains the only write path, so CLAUDE.md §5 and ADR-016's read-only tool boundary stay intact.
- SYSTEM_PROMPT and add_todo's own tool description now say that a financial correction must use the preview flow and must never be recorded as a task unless the person explicitly asked for a to-do, reminder, feature, or idea.
- 2026-08-25Credit card payment as its own subcategory and transfer signal
- Added the system subcategory credit_card_payment (Credit card payment) under Financial & fees, available everywhere the seeded taxonomy drives: classification, category pickers, filters, rules, and totals. Migration d8f1a2c3b4e5 installs it idempotently after the category reorganisation.
- A leg filed as credit_card_payment now counts as corroborating evidence for automatic internal-transfer matching, alongside txn_type == transfer. Exact opposite amount, currency, date window, and mutual uniqueness remain mandatory; the category only supplies the missing same-bank transfer signal. ADR-028 and ADR-031 record the relationship.
- 2026-08-24Select the other side of an internal transfer by hand
- An unpaired transaction marked as an internal transfer now offers a Match with another transaction selector in its editor. Candidates are owner-scoped, unpaired movements with the exact opposite amount in the same currency, and show their date, name, amount, and source bank before the user chooses.
- POST /api/transactions/{id}/transfer-match links the selected rows in both directions and marks both transfer_verified; it refuses self-matches, mismatched amounts/currencies, and rows already belonging to another pair. The human choice may cross the automatic four-day window or same-bank evidence rule, because those are safeguards for an algorithm guessing, not reasons to overrule an explicit selection. ADR-028 records the boundary.
- Saving without selecting a counterpart still permits a one-sided internal transfer while the other bank has not imported it yet. Once its counterpart exists, reopening the transaction offers it for selection.
- 2026-08-24Review matched transfers after upload, and sort the ledger by name or bank
- The Transactions page now has a persistent Review matched pairs only view. Each match is one card with its outgoing and incoming movements side by side, including both names, dates, amounts, categories, and source banks. Either side opens in the existing correction dialog; unchecking the transfer there still splits both legs and permanently protects them from being matched again. The upload summary now links directly to this view instead of to a transaction list that continued hiding transfers by default.
- Transaction ordering now covers name A–Z/Z–A and source bank A–Z/Z–A as well as the existing absolute-amount choices. Sorting happens in the repository, before pagination, and source-bank ordering follows the same provenance-then- corroboration rule as the bank name shown on a transaction.
- A transaction's editable name is now exposed in the correction dialog; the immutable raw bank wording remains alongside it. Empty names are rejected.
- 2026-08-24Categories get a real parent, and totals roll up to it
- Reparented the loose top-level categories into families: financial (bank fees, taxes, loan repayments), savings_investments (savings, investments, transfers, cash withdrawals), and split insurance into health/home/auto/life children instead of one undifferentiated bucket. Migration 5168d5d8c06c; idempotent against ensure_seeded having already inserted the new slugs from a boot that ran ahead of the migration.
- A category invented from a chat sentence (services/rules.py) is now asked to name its top-level parent, the same way it already resolves a synonym to an existing category — it no longer lands as a permanent orphan at the top level.
- totals_by_category returns each row's parent_slug; the dashboard's "Where it went" now shows parent-level totals by default (Overview.by_category_grouped), drilling down into every category in that family, with by_category kept for the leaf-level view. The totals MCP tool gained group_by="category_group" alongside the unchanged "category".
- The category <select> in AddTransaction/EditDialog renders real <optgroup>s instead of a flat list with a literal "— " prefix.
- ADR-031 records the taxonomy and the reason reparenting has to live in a migration rather than the seed list. Verified by the full 533-test backend suite and frontend type checking.
- 2026-08-24ADR-031Categories get a real parent, not just a label
- The category table has had a self-referential parent_slug since the initial schema, but nothing used it consistently. Only 7 of 17 top-level system categories had children; the rest — bank_fees, taxes, loan_payments, savings, investments, transfers, cash_withdrawal, insurance, and others — sat at the same level as spending categories like food, mixing "money you spent" with "money you moved" and "cost of managing money." Worse, every category a person invents from chat (services/rules.py, ADR-012) was created with parent_slug=None and stayed that way forever — there was no code path that ever set it — so a taxonomy that started with structure drifted toward a flat list of orphans over time. Separately, totals_by_category grouped strictly by leaf, so even the categories that did have children never showed a parent-level total anywhere.
- 2026-08-18Banks do not deduplicate each other, and a person can undo a match
- Exact-hash and amount/date reconciliation now reject a candidate sourced from a differently named bank. A first Millennium import can no longer say a Santander/Bankinter row was already in the ledger merely because its date, amount and wording happen to coincide. Bank-qualified hashes preserve legitimate overlap detection on later imports from Millennium itself.
- An automatically absorbed movement is named in the import summary with This match is wrong — keep both. The correction recreates the bank row, unconfirms the earlier row, and stores keep_separate, added by migration a91f7c2e4d38, so a later statement cannot silently merge it again.
- The transaction list can sort by newest, largest absolute amount, or smallest absolute amount. Opening a transaction now names the bank derived from its source statement (or the statement that corroborated it).
- Reset all data moved from Owners into Delete data. The current owner may reset themselves; an admin may still reset another owner through the API.
- ADR-030 records the reconciliation and correction boundary. Verified by the full 533-test backend suite and frontend type checking.
- 2026-08-18Reset one owner to an empty FinAid without removing their login
- The admin Owners screen now has Reset all beside each person. Its confirmation names the target and spells out that transactions, statements, receipts, learned rules, chats and tasks are permanently deleted.
- POST /api/owners/{owner_id}/reset is admin-only and clears every table scoped to that owner, plus their uploaded statement and receipt files. The owner row, login token, admin status, monthly limit and OAuth grants remain, so a reset person can immediately start again without reconnecting.
- The database deletion is one transaction in foreign-key order and never reaches another owner's rows or shared system categories. Files are deleted only after the commit, following the same failure preference as ADR-016.
- ADR-029 records the boundary. Service tests cover preserved identity, cross-owner isolation, file removal and the admin permission.
- 2026-08-18Fix: a transfer leg could be silently absorbed into an unrelated coincidence
- Reported after uploading a first-ever Bankinter statement: "1 was already in your ledger" / "confirmed... matched to an entry of your own", even though nothing from Bankinter had ever been imported before. The owner correctly diagnosed it: the row it merged into came from a different bank's unconfirmed import (a fragment, so confirmed_by_id was still null even though it had a real statement_id), and it only shared an amount with the Bankinter line by coincidence. The Bankinter line was actually one leg of a real transfer — confirmed by the owner because, had it been inserted instead of absorbed, it would have formed a third, correct internal-transfer pair.
- Root cause: _reconcile's pass 2 (services/ingestion.py) matches an incoming line against unconfirmed_between candidates on amount and a date window alone, with nothing else to tell two same-amount, same-window movements apart when the pairing happens to be unique on both sides — the mutual-uniqueness rule protects against multiple candidates, not a single wrong one. Before the internal-transfer feature (2026-08-18, earlier today) existed, there was no second source of evidence to catch this with.
- New IngestionService._looks_like_a_transfer_leg: before pass 2 runs, an incoming line with its own opposite-signed counterpart elsewhere in the ledger — cross-bank, or either side carrying the transfer wording signal, the same evidence _pair_transfers itself requires — is set aside and never offered to absorption. It is deliberately lenient: this only has to be good enough to keep a real transfer leg out of pass 2, not to pair it correctly — match_internal_transfers, which already runs right after insertion, does the actual rigorous, mutual-uniqueness pairing once the line is safely its own row. Excluding a line here costs nothing worse than an ordinary new transaction; wrongly absorbing one silently discards real money and misreports it as a duplicate.
- New tests/test_ingestion.py::TestInternalTransfers:: test_a_transfer_leg_is_not_wrongly_absorbed_into_an_unrelated_decoy reproduces the exact reported shape: an unconfirmed same-amount decoy from an unrelated bank sits in the absorption window, a genuine opposite-signed counterpart sits in the transfer window; the incoming line must be inserted and paired with the counterpart, and the decoy must be left untouched. 528 tests pass on SQLite; no schema or query shape changed (the fix reuses TransactionFilter/search, already proven on both engines), so no fresh Postgres run was needed.
- 2026-08-18Accept the legacy binary `.xls` format too
- CSV and .xlsx uploads shipped earlier today; .xls (Excel 97-2003, a different binary container openpyxl cannot read at all) was explicitly left out as "near-obsolete" — a judgement call, not a technical limit, and the owner asked for it because their own bank still exports it that way. See ADR-006's second amendment.
- SourceDocument.is_xls (domain/ports.py) and _xls_to_text (parsers/llm_parser.py, via the new xlrd dependency — chosen because its 2.0+ line deliberately reads only .xls, leaving .xlsx to openpyxl, so the two never overlap or disagree about which one owns a file) mirror is_xlsx/_xlsx_to_text exactly: the same per-sheet, per-row plain-text shape reaches the same LLMStatementParser call, so the extraction prompt, schema, and classification pipeline needed no change at all. .xls and .xlsx filenames are told apart by extension before either library ever sees the bytes.
- The web upload page's file picker now also accepts .xls / application/vnd.ms-excel.
- New tests/test_spreadsheet_uploads.py::TestXlsDecoding mirrors TestXlsxDecoding exactly (a workbook becomes a text table, a date cell renders as ISO, a corrupt workbook fails with an actionable message, an empty workbook is refused), plus an end-to-end ingestion test. Fixtures are real .xls bytes built with xlwt (a new dev-only dependency — xlrd itself only reads), never a real statement (CLAUDE.md #6). 527 tests pass on SQLite; no schema or query changed, so this did not need a Postgres run.
- 2026-08-18Detect and hide money moved between the owner's own accounts
- The owner holds accounts at more than one bank and regularly moves money between them — paying one bank's card from another bank's account, say. Each leg is a real, correctly-imported transaction, so nothing about the duplicate/absorption machinery applies: a -3000 EUR outflow and a +3000 EUR inflow are the same money, not one movement printed twice — but counting both as spending and income inflates every total with money that never left the owner's own pocket. TransactionType.TRANSFER already existed and the classification prompt already documented it as "must not be counted as such," but nothing acted on that: totals included it regardless, and the classifier's guess was a single-sided read of one statement's wording, so it missed anything that did not happen to say something transfer-shaped.
- A transfer is now established by matching two transactions, not by reading one. New services/ingestion.py::_pair_transfers, shaped like the existing absorption pairing (_pair_up): an outflow and an inflow of the exact same amount and currency, booked within four days of each other, mutual-uniqueness required (an ambiguous match is left alone rather than guessed). Confirmed with the owner: amount matching is exact only, no fee/FX tolerance; a different bank on each leg is evidence enough by itself, the same bank additionally requires one leg to already read as txn_type == transfer (wording like "PGMT CART" / "CRD PYMNT"), since two coincidentally equal opposite amounts at one bank are more likely an unrelated coincidence. See ADR-028.
- A match sets Transaction.is_internal_transfer and transfer_pair_id on both legs. Every query that already funnels through SqlTransactionRepository._apply — search, count, totals_by_category, totals_by_period, and therefore the dashboard, /api/transactions, and every MCP/chat tool — excludes these rows by default, one choke point rather than a filter repeated per call site. TransactionFilter.include_internal_transfers=True opts back in.
- Runs automatically at the end of every ingest (a future statement catches its own transfers immediately, whichever bank's statement arrives first) and is separately exposed as POST /api/transactions/ rescan-transfers — REST-only, like reparse and deletion, since this writes financial classification at ledger scale and tools stay read-only over financial data without exception (CLAUDE.md #5) — for catching pairs already on file from before this existed.
- A human correction is permanent, the same way user_verified already is for classification: transfer_verified, set via TransactionPatch. is_internal_transfer, stops the matcher from ever flipping is_internal_transfer on that row again. Unmarking a linked row symmetrically unmarks its matched leg too — a wrongly-matched pair is wrong on both sides — and marks both verified so they never silently re-match on a later rescan.
- web/app/transactions/page.tsx gained a "Show internal transfers between my accounts" toggle (off by default) and a "Rescan for internal transfers" action; EditDialog gained the manual override checkbox and shows the linked leg when one exists. Correcting transfer status refetches the page rather than patching the edited row in place, because the backend's symmetric unlink can also change a second row the PATCH response does not include — caught by driving the actual feature in a browser (Playwright against the dev servers), not just the test suite. web/app/upload/page.tsx gained a disclosure alongside the existing duplicate/absorbed ones, naming which rows a fresh import matched.
- Migration a7c4e29f1b06. No backfill needed or attempted: existing rows default to untouched/unmatched, which is truthful, and a rescan (or the next upload) catches pairs already on file. 518 tests pass on SQLite and Postgres 16, including new tests/test_ingestion.py:: TestInternalTransfers and tests/test_overview.py:: TestInternalTransfersAreExcludedByDefault.
- 2026-08-18Fix: "already in your ledger" was wrong, not just unnamed — two same-day, same-amount lines in one CSV are not automatically the same movement
- Looking at the actual CSV behind the report the entry below was chasing: the "2 were already in your ledger" it now correctly named were not matches against anything previously imported at all. They were two genuinely separate transactions from that same CSV — two purchases at a vending machine, same day, same price — that happened to hash identically and got silently collapsed to one by pass 0 (_dedupe_within_batch, added in the entry directly below this one). Naming the row more clearly, which is what that entry did, could not fix this: the row it was naming was correct on its face and wrong in what it claimed. The grammar wasn't the bug. The premise was — that a document printing the same date, amount and wording twice always means one printed line got transcribed twice, never two real movements that happen to coincide. It doesn't always mean that, and guessing wrong here means discarding real money.
- Pass 0 is gone. _dedupe_within_batch collapsed same-hash lines before _reconcile ever ran; it is replaced by _disambiguate_batch_repeats (services/ingestion.py), called once right after a freshly-parsed statement's Transaction objects are built. Instead of detecting and discarding a same-batch collision after the fact, it prevents the false collision from ever existing: the first occurrence of a given (date, amount, currency, wording) combination keeps today's hash unchanged, and every occurrence after it folds its position in the batch into Transaction.compute_dedupe_hash(occurrence=n) — a new optional parameter, byte-identical to the old formula at its default. Two real repeated movements now simply have two different hashes and both survive as ordinary, undisputed rows; nothing about them is reported as a duplicate, because nothing about them was one. _reconcile's pass 1 (matching against rows already on file) needed no change at all — once the false collision can't happen, there is nothing left for it to special-case.
- Kept, not thrown away: a document that genuinely does print (or a parser genuinely does transcribe) one line twice by mistake is still visible — both rows are simply kept as real transactions, and a new warning ("N transaction(s) share the exact same date, amount and description as another line in this import…") points at it, so a person can glance and delete the extra one via Delete data if that's what actually happened. This is the same "never silently guess, surface the ambiguity" rule pass 2's mutual-uniqueness check already lives by, applied to a case that pass 0 had instead resolved by guessing and discarding.
- No migration: existing rows keep the hash they already have (occurrence defaults to 0, which reproduces the exact prior formula), so nothing needs rehashing.
- Rewrote test_ingestion.py::TestDuplicates:: test_the_same_line_printed_twice_in_one_document_is_not_double_inserted — its old assertion (imported == 3, the repeat "collapses to one") was encoding the exact bug — into test_a_hash_identical_line_within_one_ document_is_kept_not_discarded (both rows survive, duplicates == 0, the new warning fires) plus test_a_second_overlapping_statement_still_recognises_both_repeats, confirming the disambiguated hashes still do pass 1's original job: a later overlapping statement reprinting both repeated lines is recognised as two prior duplicates, not reimported. 508 tests pass on SQLite.
- 2026-08-18Fix: two more ways a duplicate could go unnamed after the fix above
- Reported after redeploying the previous fix: still nothing to click on a real import (a CSV, 64 transactions, 2 duplicates). The invariant the previous fix relied on — len(duplicate_matches) == duplicates — only covered duplicates _reconcile actually looks for. Two more ways add_many (infra/db/repositories/transactions.py) silently drops a row it was never asked about: 1. Within one document. add_many already refuses a second row whose hash matches one earlier in the same insert batch — a document printing (or a parser producing) the same movement twice within itself, not a match against anything already on file. _reconcile never produced a candidate for this at all, so nothing was ever going to be in duplicate_matches for it, whatever the earlier fix did. 2. A fragment repeating something already on file. _reconcile's exact-hash pass only ran if document_kind is STATEMENT — "only a statement can corroborate" (ADR-015) was read as "only look up existing hashes for a statement," conflating confirming a row (a privilege ADR-015 does reserve to statements) with not reinserting an exact repeat (something add_many already enforced for every document kind, just silently).
- _reconcile (services/ingestion.py) now runs a pass 0 — new _dedupe_within_batch, collapsing hash-identical lines inside one incoming document before anything else, regardless of document kind — and pass 1's exact-hash lookup now also runs unconditionally; only the confirming side effect (confirm_ids) stays gated to document_kind is STATEMENT. Neither change alters what gets persisted — add_many was already preventing both kinds of double-insert — only what gets reported.
- New test_ingestion.py::TestDuplicates:: test_the_same_line_printed_twice_in_one_document_is_not_double_inserted and test_corroboration.py::TestFragmentsCorroborateNothing:: test_a_fragment_still_does_not_reinsert_an_exact_repeat reproduce the two gaps directly, alongside the existing invariant test from the prior entry. 507 tests pass on SQLite.
- 2026-08-18Fix: "nothing to click" when the duplicates were absorbed, not exact matches
- Reported immediately after the entry below shipped: no disclosure appeared under "X were already in your ledger" at all. Root cause: duplicates (services/ingestion.py) is len(transactions) - len(inserted), which counts both a pass-1 exact-hash match and a pass-2 absorbed match (one worded differently, matched to something typed by hand) as "produced no new row" — but duplicate_matches was only ever populated from pass 1. When a document's duplicates happened to be entirely absorbed ones (the common case for a hand-typed entry the statement later confirms), the list came back empty while the count stayed positive, so the frontend's duplicate_matches.length > 0 check silently fell back to plain, unclickable text.
- duplicate_matches is now built from both reconciliation.duplicate_matches (pass 1) and reconciliation.absorbed (pass 2) — the same two sources duplicates itself sums — so len(duplicate_matches) == duplicates always holds. A transaction can legitimately appear in both duplicate_matches and absorbed_matches: being an absorbed match and being "already in the ledger" are true of the same row at once, not two different facts.
- New test_corroboration.py::TestAbsorbing:: test_an_absorbed_row_is_also_a_named_duplicate reproduces the exact reported scenario (a duplicate purely from absorption), and test_duplicate_matches_always_accounts_for_the_whole_count asserts the invariant directly so a future change that breaks it fails loudly instead of silently emptying a list. 505 tests pass on SQLite.
- 2026-08-18Naming which transactions an "already in your ledger" count refers to
- Asked after an upload reported "2 were already in your ledger, so they were not counted twice": there was no way to see which two. IngestionResult (services/ingestion.py) carried only aggregate duplicates/absorbed counts; the actual rows were known internally (_reconcile already builds them) but discarded before reaching the API.
- _Reconciliation now also collects duplicate_matches — every existing row a pass-1 exact-hash match landed on, whether that row was freshly confirmed or was already confirmed by an earlier overlapping statement — alongside the pass-2 absorbed rows it already tracked. Both surface as list[MatchedTransaction] (transaction_id, description, amount_display, booked_on — the same shape PossibleDuplicate already used for the ambiguous case) on IngestionResult, mapped through UploadResultOut as duplicate_matches/absorbed_matches.
- web/app/upload/page.tsx's summary lines for "already in your ledger" and "now confirmed by this statement" are now a <details> disclosure — the count stays the headline, and expanding it lists exactly which rows, mirroring how possible_duplicates was already shown.
- web/lib/api-schema.d.ts regenerated via npm run gen:types, and web/lib/history.json/backend/src/finaid/static/history.json regenerated in step (the prior CSV/XLSX changelog entry had been added to CLAUDE.md without that step, so this catches both up together).
- New coverage in backend/tests/test_corroboration.py::TestAbsorbing (the absorbed row's id and bank-wording description are named) and an extended assertion in test_ingestion.py::TestDuplicates:: test_overlapping_statements_do_not_double_count (the three matched ids are exactly the three rows already on file). 503 tests pass on SQLite.
- 2026-08-18Accept CSV and XLSX statement exports, not just PDF and image
- The owner asked to upload a bank's CSV/XLSX export directly rather than always going through a PDF or a photo. Per ADR-006, statement parsing already goes through one LLMStatementParser precisely because banks have no common format — that reasoning applies just as much to a spreadsheet export's column layout as to a PDF's, so this extends the existing parser rather than adding a new deterministic one with per-bank column-mapping logic (see ADR-006's amendment below).
- SourceDocument (domain/ports.py) gained is_csv/is_xlsx, mirroring the existing is_pdf/is_image (content-type or filename-extension). LLMStatementParser.supports() claims both; _document_block decodes a CSV to text (trying utf-8-sig, utf-8, cp1252, then latin-1 — the last always succeeds, so this fails open like _normalise_pdf does, and covers the legacy Windows encodings some European banks still export in) and converts an XLSX workbook to a plain-text table via openpyxl (one line per row, one # Sheet: section per non-empty sheet, dates rendered ISO). Both become a TextBlock sent through the same structured-output extract() call already used for PDFs and images — no schema change, since visual_fingerprint/document_marks already have an "unknown"/empty-string escape hatch for a trait that genuinely isn't on the page, which a spreadsheet's missing logo and colours trivially is. SYSTEM_PROMPT gained matching instructions: identify the bank from printed text rather than appearance when there is none to go on, and never transcribe a spreadsheet's header row as a transaction.
- A corrupt or unreadable .xlsx (including a legacy binary .xls misnamed .xlsx — openpyxl cannot read that format, and adding xlrd for a near-obsolete one was not judged worth it) fails with an actionable ParsingError before it reaches the model, the same pattern already used for HEIC images and password-protected PDFs.
- services/ingestion.py needed no changes: source_kind already falls back to STATEMENT_UPLOAD for anything that isn't is_image, which is the correct bucket for a spreadsheet export, and the storage-key extension is already derived generically from the filename.
- Web upload (web/app/upload/page.tsx) now accepts .csv/.xlsx in its file picker (the camera input is untouched — a spreadsheet is never photographed) and its copy says "PDF, spreadsheet or image".
- New backend/tests/test_spreadsheet_uploads.py: format claiming by extension and content-type, UTF-8 and cp1252 CSV decoding, an XLSX workbook becoming a text table with ISO dates, a corrupt/empty workbook failing with a clear message, and end-to-end ingestion producing a STATEMENT_UPLOAD (not IMAGE_UPLOAD) statement. 502 tests pass on SQLite. Not verified against a live Anthropic call in this environment — the same limitation noted in earlier entries — but the code path converges on the exact extract() call already proven to work for PDFs and images, just with a TextBlock in place of a DocumentBlock/ImageBlock.
- 2026-08-18ADR-029Resetting data keeps the owner's identity
- An admin can manage data as another household member, but there was no way to return that person's FinAid to a clean state. Removing an owner is deliberately only access revocation (ADR-020), and the ordinary deletion flow is intentionally selective, so neither operation means “start again”.
- 2026-08-18ADR-030Bank identity bounds deduplication; rejected matches stay rejected
- A first import from one bank reported a movement as “already in your ledger” because reconciliation found a coincident row sourced from a different bank. Even where an automatic match to a manual entry is reasonable, the import summary offered no way to say the judgement was wrong. Separately, the whole-account reset sat beside identity removal rather than alongside the other destructive data controls.
- 2026-08-17Teaching FinAid that a printed line isn't a transaction, for good
- Reported: typing "Saldo Anterior nos extratos do Santander não é uma transação" into the feedback box only reclassified what was already on screen — the same balance-carry-forward line would be extracted again on the next Santander statement. The owner asked for this to work the way bank recognition already does (ADR-010/011): taught once, applied to every future parse — and explicitly rejected the first draft of the fix, which reused CategoryRule with an is_exclusion flag, as conflating two different kinds of knowledge. See ADR-027.
- New ParsingHint entity (domain/models.py), shaped like BankHint rather than CategoryRule: a kind ("not_a_transaction" today, room for more later), a pattern, and an optional bank scope. Own table (parsing_hints, migration b3f7a1c9d452), own repository, no category_slug or recurrence_override anywhere near it.
- Taught through the same sentence box and the same /feedback/interpret → /feedback/apply two-step as a CategoryRule — RuleService.interpret() now recognises a third action, not_a_transaction, and builds a ParsingHint instead of a rule when that's what the sentence describes.
- Applied with BankHint's two-tier shape: a soft prompt hint (_parsing_hint_text in parsers/llm_parser.py, best-effort), and the actual guarantee — IngestionService._apply_parsing_hints deterministically drops any matching parsed line in ingest_parsed, before assess_completeness runs. The ordering matters: a misread balance line typically carries the balance itself as its "amount," which would otherwise corrupt the opening + movements = closing arithmetic that decides whether a document is a complete statement (ADR-015).
- Forward-only by default: teaching a ParsingHint never touches a transaction already on file. The preview still reports how many existing transactions look like the pattern, but only as information — removing something already wrongly imported at ledger scale is what Delete data's preview-then-execute flow is for.
- One narrow exception, added after review: right after an upload, the person is already looking at exactly the statement that produced the mistake, and sending them to Delete data separately to remove something they can already see is friction the feature exists to avoid. interpret() optionally takes the statement_id in view and reports how many of that statement's own rows match; apply_interpreted() takes an explicit, default-off remove_from_statement flag that removes them — all-or-nothing per statement, and only when nothing in the matching rows was corrected by hand or has a receipt (the same guard reparse already uses for the identical question). TeachBox (used on /upload, /transactions, and /rules) shows this as a checkbox, checked by default but visible and declinable, when it applies. statement_id never narrows a CategoryRule preview, which keeps scanning the whole ledger exactly as before.
- /rules gained a second section, "What FinAid ignores when reading a document," with the same turn-off/delete controls as category rules. export_learnings/import_learnings now carry ParsingHints too, so "everything learned" stays true across a move to a new installation.
- New tests/test_ingestion.py::TestParsingHints proves the ordering point directly, not just the filtering: with an active hint, a matching line is dropped and the completeness arithmetic reconciles; without one, the same line is still extracted and the arithmetic does not — the exact corruption the ordering fix prevents. tests/test_rules.py:: TestParsingHints covers the statement-scoped removal: fires only within the statement in view, is a no-op unless explicitly requested, and is blocked rather than partially applied by a hand-corrected row or an attached receipt. 489 tests pass on SQLite and Postgres 16; the migration was verified up/down/up against a real Postgres 16 instance.
- 2026-08-17Re-serialise a PDF before it reaches the API, not just check it
- The password check above did not explain everything: reported back that a PDF downloaded directly from the bank's own site, opening without warnings in every ordinary viewer, still failed with the same "not valid" answer from Anthropic. Ordinary PDF viewers are famously forgiving of structural looseness that a stricter downstream parser is not — this is a well-known class of problem with PDFs produced by in-house banking/reporting systems, and the standard fix is to re-serialise the file into a conformant structure, not to guess at what is technically wrong with it.
- parsers/llm_parser.py::_normalise_pdf reads every page with pypdf and writes a fresh PDF back out before it is base64-encoded, for both the statement and the receipt parser. This repackages the container without touching what is on the page. Fails open by construction: anything pypdf itself cannot make sense of — including a PDF _check_pdf_readable already refused as locked, and the placeholder bytes the test suite already used — is sent through completely unchanged, so this can only ever leave an already-broken case exactly as broken, never turn a working one into a broken one.
- Unlike the reverted SYSTEM_PROMPT change earlier today, this is deterministic and testable without a live model call: tests/ test_pdf_encryption.py gained TestNormalisePdf, confirming page count survives a real re-serialisation, non-PDF bytes and a locked PDF both pass through byte-for-byte unchanged. 474 tests pass. Whether it actually fixes the reported document specifically cannot be confirmed from here — there is no live Anthropic API call available in this environment — so this is offered as a well-reasoned, low-risk candidate, not a verified fix; the practical workaround in the meantime is re-exporting the PDF through a different tool (a viewer's "print to PDF") or uploading a screenshot of it.
- 2026-08-17Fix: a password-protected PDF answered with a raw API error
- Reported from the deployed app: Anthropic API error 400: ... "The PDF specified was not valid.", verbatim, on a fresh upload. The Anthropic API cannot open a PDF that needs a real password, and its own error says nothing about why or what to do — the same wording covers a page that is not a PDF at all.
- parsers/llm_parser.py::_check_pdf_readable checks locally, before spending a model call: pypdf (new dependency, pure Python, only used here and in the receipt parser) opens the PDF and tries decrypt(""). A PDF "encrypted" with an empty user password — an owner-only lock most readers open without ever prompting — is not what a person means by "password-protected" and passes straight through; only a PDF that genuinely needs a password to open now stops with a clear, actionable message before the API is ever called. Anything that is not confidently an encryption problem (including a document that is not a valid PDF at all, or the placeholder bytes the test suite already used) is left to the API to answer, rather than risk a false rejection.
- Shared with LLMReceiptParser, since an uploaded receipt can be a PDF too.
- New tests/test_pdf_encryption.py: an ordinary PDF, a genuinely locked one, an owner-only-locked one, and non-PDF bytes, each unit-tested against _check_pdf_readable directly with real PDFs built by pypdf.PdfWriter — and end to end through ingest_upload, confirming a locked PDF never reaches the model (FakeLLM.extract_calls == []) and the statement is marked failed with the clear message rather than the API's own. Verified live over HTTP against a real encrypted PDF: 422 ParsingError with the intended message, not the API's 400. 471 tests pass.
- 2026-08-17Reverted: the previous prompt change made the duplicate worse, not better
- The "one printed movement is one transaction, even across multiple visual rows" instruction added earlier today (below) was reported back within the hour: instead of the rare single-line duplicate it was meant to fix, every card-network/miles sub-row on the statement started coming back as its own transaction, merchant "VIS", amount the miles figure — a systemic failure worse than the one it replaced.
- Reverted in full, back to the exact wording before that change (verified against the prior commit with git diff, byte-identical). The likely mechanism: describing the sub-row pattern in enough detail for the model to recognise it reliably also seems to have made it more likely to report a match, regardless of the accompanying "never transcribe it" instruction — a known failure mode where a vivid description of a pattern can outweigh a negative instruction attached to it. This is a genuine limit of tuning SYSTEM_PROMPT from this environment: there is no live model call available here to verify a wording change actually does what it says before it reaches production, unlike every other change in this codebase, which is exercised by pytest before being trusted.
- Left as the smaller, known issue for now rather than attempting another unverified prompt edit: an occasional duplicate line, correctable per document with Read this document again or by removing the extra row by hand, is a much smaller failure than a systemic one across every affected statement.
- 2026-08-17Fix: one printed line became two transactions
- The owner's example, from a real statement: one row for "COMPRA 3088 LIVESQUARE LDA LISBOA CONT", -95.00 EUR, 30/07 — and FinAid held two, -95.00 EUR each, one described with a trailing "VIS" the other did not have. Same statement, same upload, not two overlapping documents — so neither dedupe_hash (the descriptions differ, so the hashes legitimately differ) nor cross-statement absorption (there was only one statement) had any way to catch it. The parser itself produced two transactions for one printed movement.
- The document prints each movement across two rows: a main row (date, description, debit/credit) and a second, indented row underneath holding only the card network code ("VIS") and loyalty miles — no date of its own, no amount in the debit/credit column. LLMStatementParser's prompt had no rule for that shape and, on this line, read the second row as a movement of its own, inheriting the amount from the row above and picking up "VIS" into the description. Every other line on this same statement happens to have printed the identical two-row shape without being duplicated, which is exactly what made this look arbitrary rather than systematic.
- The prompt now says explicitly: a movement is one transaction even when a statement prints it across more than one visual row; a secondary row with no date and no debit/credit amount of its own is metadata about the movement above it, never a transaction, and never merged into that movement's description either — the description stays exactly what its own row printed. Two rows are two transactions only when each one carries its own date and its own amount.
- No automated test exercises this: it is a natural-language nudge to the model, not schema or pipeline logic, and this codebase has no precedent for asserting on prompt text (CLAUDE.md #5's structured-output tests check shape, not wording). The owner can re-run the statement that surfaced this with Read this document again (added earlier today) once deployed, to confirm in a real document rather than a synthetic one.
- 2026-08-17Fix: a slow-but-working upload looked like it had already been uploaded
- Reported from the deployed app: "already uploaded" on what was, from the owner's side, a single upload attempt — not a retry. Root cause was the API proxy, not the ingestion pipeline: web/app/api/[...path]/route.ts gives every request 25 seconds before aborting, and treated that abort the same as a connection that never reached the backend at all — "safe to resend even for a POST." A real statement parse (reading the document, classifying every line, more so with the "more capable model" box ticked) routinely takes longer than that once the connection is live, so the proxy gave up and silently resent the exact same file while the first attempt was still correctly importing it. By the time the resend's response came back, the original import had finished and the resend's own duplicate-file check refused it — which is a correct answer to a question nobody actually asked.
- Non-idempotent requests (POST/PATCH/PUT/DELETE) now get 110 seconds per attempt instead of 25, and — the actual fix — are never resent on a timeout. AbortSignal.timeout() firing is distinguished from a genuine connection failure by error.name === "TimeoutError" (verified against a real Node HTTP server that accepts a connection and never answers, not assumed): a true connection failure still retries safely for any method, because the request never left this process; a self-imposed timeout does not, because the backend may already be finishing. That case now answers its own honest 503 StillProcessing instead of silently duplicating the write.
- The upload page treats that answer as "check back", not as a failure: it refreshes the statement list right away, since the import may already be sitting in it, and colours the notice the same non-alarming way an actual duplicate already was rather than as a red error.
- No JS test runner exists in web/ yet (CLAUDE.md's frontend checks are typecheck + check:contract + build), so this was verified with a standalone Node script reproducing the exact scenario — a server that accepts the connection and never responds — confirming AbortSignal. timeout() rejects with TimeoutError, and separately that an unreachable port rejects with a plain TypeError, which is the distinction the fix depends on.
- 2026-08-17Read a statement again, without deleting and re-uploading it
- Reported from the deployed app: a low-confidence import gave no numeric answer to "did it actually work", and the only way to try again with the better model was to delete the import and upload the exact same file a second time — which the duplicate-file check then had every reason to refuse, reading as though the first attempt had failed.
- POST /api/statements/{id}/reparse re-reads the file already held in storage — no re-upload — with a quality choice and an optional free-text feedback ("you imported a lot of duplicates") that reaches the parser's own prompt for that one attempt. It is REST-only, never a chat or MCP tool: it deletes and recreates data, and tools stay read-only over financial data without exception (CLAUDE.md #5).
- Refused, not silently overwritten, when it would lose something. If any transaction the statement produced has since been corrected by hand or given a receipt, IngestionService.reparse raises ReparseBlockedError (409) naming how many, and points at the existing Delete data flow instead — which previews before it removes anything. Otherwise it wipes exactly what the statement produced (never what it merely confirmed — those revert to uncorroborated via the same detach_from_statements call DeletionService already uses) and re-runs the full pipeline.
- UploadResultOut now carries average_confidence, the classifier's own mean confidence across the newly inserted rows — an actual number where the low/normal quality heuristic was previously the only signal. The upload page shows it, and re-uploading the exact same already-imported file now redirects straight to what it produced instead of showing a banner that reads as an error.
- StatementParser.parse() gained an optional feedback parameter, one implementation (LLMStatementParser) to update. Not stored anywhere: unlike a taught rule, it applies to this one re-read and nothing after it.
- New TestReparse in tests/test_ingestion.py: re-reading a stored file without doubling the ledger, feedback reaching the model's prompt, refusal when a row was corrected by hand, refusal before a first successful import, and an unknown statement id. 464 tests pass on SQLite and Postgres 16.
- Not done. The duplicate-transaction report that prompted this — two overlapping statements producing a differently-worded copy of the same movement, which today's exact-hash-or-unconfirmed-only matching cannot catch because the existing row is already confirmed by the earlier statement — is diagnosed but not yet fixed: waiting on a concrete example from the owner before changing _reconcile's matching, since extending it to already-confirmed rows needs care about which statement's wording wins.
- 2026-08-17Fix: a target category taught in another language spawned a duplicate, and matched nothing
- Reported from the deployed app: "livesquare é sempre médico" created a new "médico" category instead of reusing the existing "Medical & dental", and the resulting rule matched zero transactions even though an uncategorised LIVESQUARE charge was sitting right there.
- Root cause was one field doing two jobs. matches_category was documented for both "the person is naming the target category" and "the person is describing transactions already filed under an existing category" — two opposite things. For this sentence the model (correctly) recognised "médico" as the existing "medical" category, but the only field available to say so was the matching-condition one, so the rule ended up requiring category_slug == "medical" on transactions that were still uncategorised — matching nothing — while a brand-new "médico" category was still created as the assignment target.
- Split the two meanings into separate schema fields. existing_target_category is the assignment target when the person's word — in any language — already means an existing category ("médico" / "consulta" → the seeded "medical"); matches_category is now documented as the matching condition only ("the insurance ones between 30 and 40" = transactions already filed under Insurance). services/rules.py::_to_rule resolves the target through the existing category's own slug and label when one matches, so the same concept is never split into one category per language it happens to get taught in, and never invented as new when it already exists.
- The interpretation prompt now carries a worked example of exactly this case, since the two fields being trivially confusable is what caused the bug in the first place.
- New regression test in tests/test_rules.py reproducing the exact scenario end to end: the rule resolves to the seeded medical slug, no medico category is created, and the LIVESQUARE transaction is matched and recategorised. 460 tests pass.
- 2026-08-17Recurring, semi-recurring, or neither
- is_recurring was a boolean set automatically the moment a merchant repeated a few times, with no regard for how much the amount actually varied — so a weekly Pingo Doce run was marked recurring exactly like Netflix, and the dashboard's "Recurring, approx. per month" figure averaged a swinging grocery bill into what read as a fixed commitment. Replaced with RecurrenceLevel (recurring / semi_recurring / none, ADR-026): recurring is a stable, predictable amount — utilities, subscriptions, loan payments, salary, and non-monthly commitments like an annual insurance premium or a quarterly tax payment just as much as a monthly bill; semi_recurring genuinely repeats but the amount swings a lot (groceries, fuel); none is everything else.
- domain.models.classify_recurrence_level does the arithmetic: below RECURRENCE_MIN_OCCURRENCES sightings it's none; above it, recurring if the group's amounts stay within 20% of their own average, semi_recurring otherwise — a spread test on the ledger's own numbers, never a category allow-list and never a test against calendar-month alignment.
- The classification prompt now proposes a recurrence_level per transaction, trusted immediately rather than forced to "not recurring" until the ledger catches up — otherwise an annual premium would need three years of statements before FinAid recognised it. IngestionService. _confirm_recurrence still has the final, correcting word once a merchant has repeated enough times, in either direction.
- The dashboard's recurring_monthly_minor estimate now sums only RECURRING groups, so groceries and fuel no longer inflate a number meant to read as a fixed commitment. find_recurring and search_transactions expose the level so chat and MCP can tell "what are my fixed monthly costs" from "what do I keep buying". The rule-teaching sentence and EditDialog both gained the third option.
- Migration c7a2e9f4b813. Verified: 459 tests pass on SQLite and Postgres 16, including new tests/test_recurrence.py covering the variability arithmetic directly, and end to end against a running instance — posting and correcting a transaction, filtering /api/transactions?recurrence_level=recurring, and /api/overview reflecting it.
- 2026-08-16Where you are on the dashboard, a "last month" preset, and hiding income by default
- Drilling into a month bar used to leave the period picker showing its old selection ("Last 3" still highlighted) next to one month's figures — the picker is now replaced by a breadcrumb (← Last 3 months › Aug 2026) while drilled down, and the spend card says which month it means, so what is being shown is never ambiguous.
- Added Last month as a fourth period pill — a complete past calendar month, unlike This month's month-to-date.
- /transactions gained an "Only show expenses" toggle, on by default: TransactionFilter.only_outflows already existed for the dashboard's own aggregates and is now exposed as an /api/transactions query parameter too. Off by default only when arriving from the "needs review" banner link — hiding income there would make the list show fewer rows than the banner promised.
- 2026-08-16The dashboard answers more than "the last 12 months"
- The Overview page's fixed ?months=12 window is now a period picker — pills for Last week / This month / Last 3, an Other native <select> for Last 6 months / Year to date / All time / Custom range… — with no new npm dependency; every chart on the page stays hand-rolled CSS, as before.
- Each stat card gets a comparison badge ("▲12% vs last period") against the window immediately preceding the selected one, of the same length — one rule, so a week, a month, three months and a custom range all compare the same way, with no per-preset special-casing. "All time" has no window before it, so its badges are simply absent rather than computed against nothing. totals_by_period gained an ungrouped "all" granularity to total an arbitrary range once, reused for both the selected period's own totals and the comparison window.
- GET /api/overview drops months for date_from/date_to; nothing else called it, so there is no compatibility shim. OverviewOut now carries the resolved window, that window's totals, and the comparison window's totals.
- Clicking a month bar re-scopes the whole page to that month using the exact same fetch as picking a period — not a second, cross-filtering data model — with "← Back to \<period\>" to return. Clicking a category bar now links to /transactions?category=…&date_from=…&date_to=…, reusing the deep-link filters /transactions already reads (the dashboard's "needs review" banner link was the existing precedent for this).
- A dedicated side-by-side comparison view was considered and deliberately deferred — noted in docs/ideas.md rather than built now, since the badges answer the common case.
- New backend/tests/test_overview.py covers the comparison-window arithmetic, including a window crossing a leap-year February and a year boundary, and the "all time" and empty-previous-period cases. Verified: 454 tests pass on SQLite and on Postgres 16, and the flow was driven in a real browser at phone width — every pill, the "Other" menu, a custom range, a month-bar drill-down and back, and a category-bar click landing correctly filtered on /transactions.
- 2026-08-16Fix: opening the app could answer "Request failed (429)"
- The dashboard fires overview and categories in parallel on load. Against a Render free instance that has gone to sleep, that pair of simultaneous requests could land while the platform was still waking it and get answered with 429 Too Many Requests instead of the 502/503/504 the proxy already knew to retry — so the user saw a bare failure on what was, from their side, simply the first visit of the day.
- web/app/api/[...path]/route.ts's cold-start retry now also covers 429, using the same backoff schedule already used for gateway errors. FinAid's own backend never raises a 429 anywhere in api/errors.py (its mapped codes are 400/403/404/409/415/422/502/503), so treating every 429 as "still starting" is safe — there is no legitimate one to mask.
- 2026-08-16Teach whether a merchant is recurring
- The feedback box shown when an uploaded statement is opened now understands recurring and non-recurring preferences as well as categories. “Por omissão, considera o Pingo Doce non-recurring” previews the matching movements, asks for confirmation, updates existing matches and applies deterministically to future uploads.
- Recurrence preferences live beside category rules, can be switched off or deleted under What FinAid has learned, and never masquerade as a category. Migration f6d2a41c9e73 makes a rule's category optional and adds its recurrence decision.
- 2026-08-16Make MCP results feel like FinAid, not a database inspector
- The portable MCP App now has dedicated views for transactions, summaries, imports, recurring spending, receipt items, tasks, history and deletion review. It presents useful names, dates, amounts, status and lightweight spending bars instead of exposing kind, rows, UUIDs and other transport details to the user.
- Contextual actions use plain labels such as Manage imports, Open tasks and Review request. Add transaction only appears where it is relevant. The View supplies its own single surface and asks hosts not to add another border, avoiding the nested-card appearance seen in secondBrain.
- A spending question gets a direct, visual answer — for example, “You spent €175.19 across 8 transactions” — rather than accounting fields such as net, inflow and minor units. The View follows the browser language (Portuguese or English) and offers View by category or the contextual FinAid page next.
- 2026-08-16Never invent the year on an imported transaction
- Banking-app screenshots often print only day and month. The extraction schema previously forced a complete ISO date without recording whether the year existed on the page, so a model placeholder such as 2020 silently became ledger data and made current spending disappear from the dashboard.
- Every extracted movement now says whether its year was printed or resolved from a printed statement period. Without one, FinAid deterministically uses the most recent occurrence that is not in the future; a printed cross-year statement period still takes precedence.
- The parser receives today's date from the application clock, includes it in the extraction context, and records a visible warning whenever it resolves missing years. Existing legitimate historical data is never rewritten by a heuristic; an incorrectly imported document must be removed and re-uploaded.
- 2026-08-16Somewhere to put a smaller idea than roadmap.md wants
- docs/ideas.md — a running list for things noted in passing during a conversation, not yet decided or scoped. roadmap.md stays for the big Phase 2 candidates with their own seam and data model; this is for everything smaller, so a good idea mentioned once has somewhere to land instead of needing a production database write it has no way to make. First entry: giving the MCP Apps View hand-rolled SVG charts for a couple of high-value tools, without breaking its "no network access" property.
- 2026-08-16Verified MCP Apps against a real client, and found the honest gap
- Tested against the actual @modelcontextprotocol/inspector CLI, not only FinAid's own test suite — its --app-info probe (built for exactly this question) confirmed all ten tools correctly advertise an app, and a real tools/call returned content, structuredContent and the contextual links together, correctly.
- What it isn't doing yet, and why that's fine today: the extension capability FinAid registers on its low-level Server object never reaches a client over initialize — not a bug, but the protocol's own design. The SDK's docs say so plainly: extension capabilities now ride a separate 2026-07-28-era method, server/discover, that a legacy initialize handshake — what every real client negotiates today, Inspector included — "has nowhere to put." The path that actually matters, each tool's _meta.ui.resourceUri, is the one just verified working. Recorded as future work in ADR-024, not fixed now, since the 2026-07-28 spec is still a release candidate and nothing depends on server/discover yet.
- 2026-08-15Let MCP clients discover FinAid's interface too
- Every tool now advertises the official MCP Apps resource ui://finaid/results/mcp-app.html. The self-contained, sandboxable View renders the same structured display data FinAid's own chat uses and keeps a normal text fallback for hosts without MCP Apps support.
- The View can continue in chat or ask the host to open contextual FinAid web pages through ui/open-link, including the Add transaction form. Dates, categories, text and statement filters travel in the URL; credentials and owner identity do not.
- /transactions now honours those deep-link filters and ?add=1. MCP exposes the View through ordinary resources/list and resources/read, advertises io.modelcontextprotocol/ui, and returns structuredContent without creating a second tool registry. See ADR-024 and docs/connecting-agents.md.
- 2026-08-15Keep the orchestrator and its user as separate identities
- secondBrain is an OAuth client, not an owner. Its authorization request can now carry login_hint=mariajoaquina; the consent page shows both names, and approval binds the grant to a mariajoaquina owner while secondBrain remains the entry under Connected agents.
- An admin approving the connection reuses a case-insensitive owner-name match or provisions a new non-admin owner. A non-admin may only approve a hint matching their own name. The hint never grants access by itself.
- This gives multiple users of one orchestrator separate FinAid ledgers now, while ADR-023 and the existing OIDC To-do retain the honest limitation: login_hint is admin-confirmed naming, not signed identity proof.
- 2026-08-15Select an owner to manage their ledger
- More → Owners → Manage data lets an admin choose whose ledger the web app is showing, then upload statements or enter transactions for that owner without knowing their permanent token. A persistent “Logged in as” banner makes the active owner visible; changing profile stays under More rather than presenting the selected person's own data as somewhere to “return” from.
- Switch account shows that same owner roster when the current browser is authenticated as an admin. Token entry remains below it for changing the actual credential; choosing an owner changes the managed ledger, not which secret the browser holds.
- The selected id lives in an httpOnly cookie. The web proxy removes forged browser headers and adds the selection server-side; the backend still resolves the real bearer token first, requires an admin, and verifies the target exists. An owner id alone grants nothing.
- Identity-management and OAuth-grant routes always run as the real admin, and login/logout clear any selection. See ADR-022.
- Added the final solution to the live FinAid To-do list: use secondBrain as an OIDC identity provider and map (issuer, subject) to a FinAid owner without sharing or duplicating passwords.
- 2026-08-15Paste one URL; authorize the agent in the browser
- FinAid now implements MCP OAuth discovery and an authorization-code flow with PKCE. An OAuth-capable client needs only the public /mcp URL; it discovers the web authorization server and opens the browser for consent.
- Browser approval binds the grant to the signed-in owner. Access tokens live for one hour, refresh tokens rotate, all credentials are stored only as hashes, and every token is audience-bound to the exact MCP resource URL.
- More → Connected agents lists and revokes grants without removing an owner or touching financial data. Legacy owner bearer tokens remain valid during migration, but OAuth access tokens work only at MCP.
- Migration d4a9c2e71f30; PUBLIC_API_URL and PUBLIC_WEB_URL are now required deployment facts. The Render blueprint carries the known public origins explicitly, so production cannot silently advertise localhost. See ADR-021.
- secondBrain OAuth client 1.0 compatibility: the API also exposes /.well-known/oauth-metadata, /authorize, and /token. Its name-only requests are translated into the same registered-client, browser-consent, PKCE and rotating-token flow rather than maintained as a parallel protocol.
- 2026-08-14Signing in as a specific owner, and an admin who can see and revoke
- The gap ADR-019 left the same day. Named owners existed in the backend, but the web app's proxy still attached one fixed token from its own environment variable to every request — so the browser could never be more than one person. /login sets an httpOnly cookie the proxy now prefers over FINAID_API_TOKEN; nobody has to sign in for the app to keep working exactly as before, this is what makes doing so optional.
- Any owner can add another, over POST /api/owners — deliberately not admin-only, because a fresh owner starts with no data, so provisioning one grants the creator nothing of anyone else's. What is admin-only: seeing the whole roster and revoking someone. The first owner ever registered on a given FinAid is that admin, permanently, from the moment they exist.
- Removing an owner revokes a login, not data. Their transactions and statements are untouched — real deletion already has its own preview-then-execute flow (ADR-016), and this does not reach for it. Removing the last admin is refused outright, so the roster can never end up unmanageable.
- Still not a chat or MCP tool. Provisioning and removal are REST only, called by an orchestrator's own code or by a person in More → Owners — never something a conversation can trigger, the same boundary ADR-016 drew around writing identity in the first place.
- Migration b8f3e6a1d4c9. Verified end to end in a real browser, not only against the test suite: bootstrapped the first owner from open mode, signed in, added a second owner, confirmed she saw no roster and no tile, removed her as admin, and confirmed removing the last admin is refused with the reason shown on screen. 432 tests pass on SQLite.
- 2026-08-14Named owners, so more than one person can use this FinAid
- The seam ADR-005 left is now real. current_owner_id — still one function, still the only thing that changed — resolves a bearer token against an owners table instead of returning a fixed default. No owners registered stays open mode, exactly as before; the first registered owner switches the whole API and /mcp to requiring a token that resolves to one of them.
- Prompted by a concrete case, not a hypothetical one: a second person is going to use the personal-assistant orchestrator being built above FinAid, and needs to see her own transactions there, not the owner's — the multi-user schema ADR-005 always had, now with a way to actually tell two callers apart.
- No login screen, still. scripts/manage_owners.py add "Name" is the only way to register someone — prints a token once, which is the only time it is ever visible; only its SHA-256 hash is stored, matched by an indexed lookup rather than a comparison, which sidesteps the timing-attack concern secrets.compare_digest existed for on the single shared token this replaces.
- One function, two transports. services/owners.py::resolve_owner_id is the entire mechanism; the REST API and the MCP endpoint (a mounted ASGI app outside FastAPI's Depends, so it resolves the owner by hand per request via a contextvar) both call it, so "who is this" has one answer rather than two that could quietly disagree.
- Migrating an existing deployment costs nothing. If FINAID_API_TOKEN is set when the migration first runs, it seeds exactly one owner from it, so the shared token already in use on Render keeps answering to it unchanged — discovered as a design requirement while writing the migration, not patched in after breaking it.
- docs/connecting-agents.md's starting prompt now tells a new orchestrator session that FinAid supports named owners, not one shared credential, so it asks the right question before assuming a single token covers every person using it.
- Migration a4d97c25e8b1. 413 tests pass on SQLite. Not run against Postgres in this session — no Postgres instance was reachable from the environment the change was made in; the migration and queries use only portable SQLAlchemy constructs (String, DateTime(timezone=True), UniqueConstraint, select), consistent with CLAUDE.md #3, but this is named here rather than left silently unverified.
- 2026-08-14History, reachable by more than the web page
- list_history — a tool, not just a page. The same changelog and ADRs /history already renders are now reachable over chat and MCP, so an orchestrator sitting above several agents can ask each one for its history and compile a single timeline, tagging entries by which agent answered.
- Deliberately not owner-scoped. A changelog describes the software, not any owner's money, so the tool skips ctx.owner_id entirely rather than pretending there is data to protect.
- The backend cannot read this repository's root at deploy time — Render builds the API from dockerContext: ./backend alone (ADR-018) — so the same generator that already writes web/lib/history.json now writes a second, identical copy into backend/src/finaid/static/. Both are committed. Same discipline as api-schema.d.ts: generate locally, commit, ship.
- docs/connecting-agents.md now states the contract other agents are expected to match — same tool name, same JSON shape — so the orchestrator's merge logic does not need to special-case FinAid.
- 400 tests pass on SQLite.
- 2026-08-14ADR-018History is generated once and shipped to both deployables
- The owner wants one combined timeline across FinAid and the agents that will join it later: "each keeps their own, and the orchestrator asks each and compiles." That requires FinAid's own history to be reachable as data, not only as the /history web page — the same way every other capability crosses that boundary, over MCP.
- 2026-08-14ADR-019Named owners, resolved from a bearer token
- ADR-005 built the schema for multiple owners from day one — every table carries owner_id, every repository method takes it first — but left current_owner_id returning one fixed default, on the reasoning that retrofitting the schema later would touch every query while retrofitting who is asking would touch one function.
- 2026-08-14ADR-020Provisioning owners without a human running a script, and an admin who can see and revoke
- ADR-019 gave FinAid named owners, but left two real gaps once the design met its actual use:
- 2026-08-13Naming the layers, and a page that remembers
- FinAid is the umbrella; ExpenseAId is what has been built (ADR-017). Transactions, statements and receipts are one finance sub-agent, and BudgetAId and StockAId are its siblings rather than extensions of it. FinancialAgent stays thin.
- Almost nothing was renamed, which is the point: the finaid package, the database, FINAID_API_TOKEN and the URLs were always right for the umbrella. The name was doing two jobs, not the wrong job.
- What did move: the to-do list and deletion requests are now in agents/housekeeping/. Neither is a finance question, and leaving them beside expense/ would have made them look like peer domains.
- /history — a timeline of how FinAid got here, reachable from the bottom of More. Generated by npm run gen:history from this changelog and the ADRs, so a new entry appears with no second copy to keep in step. Changes and decisions on one line, filterable, because "what it started doing" and "what was ruled out and why" are different questions and the second ages better.
- Sub-agents stay packages in one deployable rather than separate services: BudgetAId needs ExpenseAId's categories, rules and totals, and across a network that becomes a duplicated vocabulary that drifts. The ports layering keeps a later split cheap.
- 2026-08-13ADR-017FinAid is the finance umbrella; ExpenseAId is what has been built
- The owner is building a wider personal assistant with FinAid as one part of it, and wants to add schedule, diet, budget, goals, stocks and crypto. Working out where each belongs exposed a naming problem: "FinAid" was being used for two different things at once — the finance domain as a whole, and the specific agent that reads statements and classifies transactions.
- 2026-08-12Deleting things, and who is allowed to
- More → Delete data. Tick what to remove — transactions, statements and screenshots, rules you taught it, banks it has learned — narrow it by period, bank, category, or to only what no statement confirms, and press Show me what this covers. Nothing happens until you have seen the count, the total, and a few examples.
- "Permanent" means two things and the preview says which. Movements read from a statement come back by uploading the file again. Receipts, anything you typed, and everything learned do not.
- Deleting a statement asks whether to take its movements. The ones it merely confirmed — a lunch you typed that it later absorbed — are never deleted. They go back to being uncorroborated, which is the truth once the document is gone.
- The assistant can prepare a deletion and cannot perform one (ADR-016). A request from chat or MCP lands in a queue on the same page, showing what it would remove now rather than when it was asked. Verified over MCP: the tool answers "NOTHING HAS BEEN DELETED", the ledger is untouched, and the row waits for approval. The port a tool holds has no method that deletes, so this is a capability it lacks rather than a rule it follows.
- Found while building: a surviving movement must be detached from a deleted statement on both columns, not just confirmed_by_id — the foreign key on statement_id catches it otherwise.
- Migration e93b6d1c8f27. 389 tests pass on SQLite and Postgres 16, including a parametrised one asserting that every filter combination previews exactly what it removes.
- 2026-08-12A to-do list, and the line an agent may not cross
- Somewhere to put "make the dashboard show coverage" at the moment you think of it. /todo, reachable under More — the phone's bottom bar holds five things, so what FinAid has learned, the list, and (next) deleting data now sit behind one tab.
- The assistant can read it and add to it, from chat and over MCP. These are the only tools in FinAid that write anything, and the exception is carved narrowly: not financial data, and adding only — ticking off and deleting stay with the person looking at the list. Each row records which surface added it, so a list that grew on its own says who grew it.
- ToolContext gained surface and a TodoWriter port — a port and not the service, because agents/ may depend on domain/ and nothing else.
- ADR-016 also settles how deletion will work, before any of it is built: the assistant may propose a deletion but never perform one. The owner asked for a WhatsApp permission check for requests from outside the UI; that cannot be written as described, because the API cannot currently tell the web app from an agent — both send the same token. So the check is not which surface but did a human see what was about to go, and WhatsApp becomes a second channel onto that approval queue rather than a separate mechanism.
- Migration c8e21f7a4b60. 347 tests pass on SQLite and Postgres 16.
- 2026-08-12ADR-016What an agent may write, and what it may only propose
- Two features arrived together, and they pull in opposite directions.
- 2026-08-11Fix: "106,80 €" lost the whole receipt
- Reported from the deployed app. A receipt photographed in Portugal prints 106,80 €, the model transcribed it faithfully, and to_minor refused it: it only ever swapped a bare comma for a point, so the euro sign made the whole string an invalid decimal and the parse failed outright.
- Amounts are now read as printed: either decimal separator, thousands separators, currency symbols and codes, any kind of space including the non-breaking one, a leading or trailing minus, and parentheses for a negative. domain/money.py still holds the only conversion, and no float is involved at any point.
- The one guess is that three digits after the final separator mean a thousands group — "1.234" is one thousand two hundred and thirty-four — except after a bare zero, because nobody writes "0.005" to mean five. Getting that wrong is how the fix first broke to_minor("0.005") == 1, which the existing rounding test caught.
- The same latent bug was in the statement parser: 1.234,56 would have failed identically, and so would 200,00 € typed into the feedback box. One fix, three call sites.
- A document-level total or balance that cannot be read is now a warning rather than a failure. It is one figure at the bottom of a page and often the blurriest thing on it; losing a whole basket of lines over it is the wrong trade. For a statement it also means the completeness arithmetic simply does not run, so the document falls back to being judged on its printed marks — conservative, which is correct.
- The web form's parser was changed in step and its behaviour matched case-by-case against the backend's, because a form that disagrees with the server shows one figure and saves another.
- 2026-08-10A screenshot is not a statement, and cash is still money
- Add a movement by hand. Cash, a transfer that has not posted. It counts in every total immediately; what it lacks is the bank's word for it.
- Uploads are judged, not assumed. A photograph of a banking app's list is recorded as a fragment: its movements are kept, but the period is not treated as accounted for. A document proves it is a statement when its opening balance plus every movement equals its closing balance — arithmetic, not the word "extrato" and emphatically not its dates. Statements running 15th to 14th are ordinary here; nothing anywhere tests a period against a calendar month.
- When the real statement arrives it absorbs what you already had. Same wording matches by hash; different wording ("Lunch 12.50" against "RESTAURANTE X LISBOA") matches on exact amount within four days. The row survives — same id — so the receipt, the note and the category you chose come with it, while the bank wins on the money and the wording.
- It refuses to guess. The pairing has to be unique in both directions, or nothing is absorbed and both candidates are reported. Two €12.50 coffees in one week would otherwise have the first silently swallow the wrong row.
- Two bugs this surfaced: apply_transaction never wrote dedupe_hash back on update, so an absorbed movement would have been re-imported as new by the next statement; and the parser was told to return nothing for a document that "is not a bank statement", which would have made every screenshot come back empty.
- "Confirmed" in the UI now means the bank vouched for it. What user_verified means is "checked by you" — a row reading "not on a statement" beside "confirmed" said two opposite things with one word.
- Migration b7c40d19e5f3. Verified: 300 tests pass on SQLite and Postgres 16, the migration applied over a database holding existing statements and transactions with a truthful backfill and downgraded cleanly, the schema matches the ORM on both engines, and the flow was driven in a browser at phone width — type a dinner, import the statement, watch one row absorb the other.
- Not done: nothing computes period coverage yet. The data supports it (union the statement intervals), but a total over a patchy period still looks exactly like a total over a complete one. See ADR-015.
- 2026-08-10ADR-015A transaction belongs to a statement, but you cannot always know which
- Until now every transaction arrived by parsing an uploaded document, and every uploaded document was assumed to be a bank statement. Two things broke that assumption at once:
- 2026-08-09What was actually inside a 150 euro charge
- Open a transaction, photograph the bill or upload it, and the lines come back: wine, arroz de pato, ovos com farinheira. Editable, because the parser transcribes what the shop printed rather than guessing what "VNH DOURO RES 75CL" means — and correcting it is what makes searching for "vinho" find it.
- The charge never changes. 150 EUR of Restaurants stays 150 EUR of Restaurants in every chart; items live inside it. That is ADR-014, and it is the whole reason the feature is safe: a receipt can be partial, misread or missing without any figure in the app disagreeing with the bank.
- A receipt that totals less than the charge (a tip, a split bill) says so plainly instead of being treated as an error or quietly reconciled.
- Two new tools, so this reaches chat and MCP alike: summarise_receipt_items ("how much on wine this year" — a figure, not a basket) and break_down_transaction (the contents of one charge). Both state that they cover only transactions with a receipt, so a number is never mistaken for total spending.
- Amounts unsigned (the sign is the transaction's), quantities stored as text ("1,234 kg" is a measure, not a float), the extraction schema at zero union-typed parameters.
- Migration d2f81b45c6a9. Verified: 258 tests pass on SQLite and Postgres 16, the migration applied over a database holding existing transactions and downgraded cleanly, the schema matches the ORM exactly (compare_metadata reports no differences on both), and the flow was driven in a browser at phone width — attach, correct a line, then ask over MCP and get the figure.
- 2026-08-09ADR-014Receipts belong to a transaction, and their items are not spending
- A bank statement records RESTAURANTE X — 150,00 and stops there. That is enough to answer "how much did I spend on restaurants" and nothing else. The owner asked for the level below it: open a transaction, photograph the bill, and see that the 150 euros was wine, arroz de pato and ovos com farinheira.
- 2026-08-06A lock on the API, and FinAid as a tool provider over MCP
- FINAID_API_TOKEN guards every /api/* route except health, and /mcp. Not enforced when unset — see ADR-013 for why a phone-configured host must not brick itself — but render.yaml generates one, /api/health reports authenticated, and startup warns when it is missing.
- The web app's proxy attaches the token server-side, so it never reaches the browser. Both services need the same value.
- /mcp exposes the existing ToolRegistry so a wider personal assistant can use FinAid as one of its agents. The tools are not redefined for the protocol: one registry, no second list to drift. Ask the calendar agent for the dates, then ask FinAid for spend per category in them — parameters in, conclusions out, and the ledger never crosses between agents.
- Two things found only by running it: the MCP SDK defaults to a localhost-only Host allowlist, which would have made the endpoint fail on any real deployment; and /mcp without a trailing slash 307s, which MCP clients do not follow on a POST.
- 209 tests pass on SQLite and Postgres 16.
- 2026-08-06Fix: a rule could not name a category, only guess at wording
- "Insurance between 30 and 40 euros assume it's pets insurance" matched nothing. The statement is Portuguese — "DD FIDELIDADE COMPANHIA DE SEGUROS" — so the English word never appears in any description, and the model had no way to say "the ones already filed under Insurance".
- Rules gained a category_is condition, and the interpretation prompt now includes the owner's category list with an explicit instruction never to translate a category name into a description fragment. A matches_category naming a slug that does not exist is discarded rather than stored.
- A rule matching zero transactions is now flagged in the preview rather than stated in passing: it almost always means the sentence was misread.
- Migration a91d3f60c5e8. 176 tests pass on SQLite and Postgres 16.
- 2026-08-06Corrections to categories are learned, not one-off edits
- Two ways to give feedback, both producing the same CategoryRule (ADR-012): - A sentence — "consider the 200 euro ATM transactions as Nina's salary and all 940 eur movements as MBA". Structured output turns it into conditions; /feedback/interpret previews what it would change and /feedback/apply commits, re-reading the sentence server-side. Offered right after an import, on any statement you open, and on /rules — the import summary disappears as soon as you navigate away, so it cannot be the only place. - The category dropdown — changing a category also teaches the merchant, keyed on recurrence_key. On by default; untick to change one row only.
- Rules are applied by RuleService deterministically, retroactively when created and on every future import. Precedence: an individually edited row (user_verified) > a rule > the model. Skipped rows are counted and reported, never silently overruled.
- A name the user invents becomes a real user category, so it appears in the totals and the dropdown rather than only inside a rule.
- A rule too broad to be safe is refused: at least a description fragment or an amount is required, or it would relabel half the ledger.
- Everything learned is listed at /rules with the sentence that created it, and can be turned off or deleted — "unless the user says otherwise".
- Migration f4c7e18b93a2. Verified: 172 tests pass on SQLite and Postgres 16, and the flow was driven in a browser end to end — teach, then import a new statement and watch it obey.
- 2026-08-06Fix: a failed upload could never be retried
- The duplicate check refused any file whose hash was already on record, including one whose parse had failed. The record was deliberately kept "so the file can be re-parsed", but no re-parse route existed and re-uploading was blocked, so a failed document was stuck for good — and the message said "already uploaded", which reads as though it worked.
- A statement that imported nothing and is not parsed can now be retried by uploading it again: the existing row is reused (id kept, error cleared) rather than a second one inserted. Anything that did import is still refused, which is what stops an overlapping statement being counted twice.
- Covers the stuck-parsing case too, which free hosting causes by stopping a service mid-request.
- The failure reason is now shown on the row. It was being stored and never displayed, so "failed" was all the user ever saw.
- 2026-08-06ADR-012Category corrections become rules, not one-off edits
- Changing a transaction's category protected that row (user_verified) and taught the system nothing. The classification prompt is built fresh each time from the category list and the batch being classified, so next month's identical merchant was categorised from scratch. Where the model was consistently right that looked fine; where the owner disagreed with it, they disagreed with it again every month.
- 2026-08-06ADR-013A shared token, and FinAid as a tool provider over MCP
- The owner is building a personal assistant of which FinAid is one part. In their words: the assistant should ask the calendar agent when the holiday was, then ask FinAid what was spent per category in those dates — without the ledger crossing between them.
- 2026-08-05Fix: the extraction schema exceeded the API's union limit
- The appearance fields added below pushed the statement schema to 21 union-typed parameters against a hard limit of 16, and the API rejected every upload with a 400. Absence is now expressed in band — empty string for text, an "unknown" enum member for the closed vocabularies — which takes the schema to 10 and leaves the nullable budget to the fields that need it.
- logo_position uses "absent" rather than "none" for "no logo on the page": "none" collided with the placeholder filtering that discards a model saying it could not tell, silently losing a real trait.
- Added a test asserting the limit across every schema the app sends. The suite could not have caught this before — FakeLLM accepts any schema, so only the real API ever saw it. See CLAUDE.md #5.
- 2026-08-05Recognising a bank by more than its account number
- A correction is now learned from everything a person would use, not just the account number. Two kinds of signal, trusted differently (ADR-011): - Printed marks — website, support number, legal entity, registration number, wordmark. Exact string matches after normalisation, so they are facts. Stored as ordinary BankHint rows, and they outrank bank_code, because a sub-brand shares its parent's institution code but not its website. - Appearance — language, palette, typeface, logo position, layout, date and decimal format, each from a closed vocabulary so two documents can actually be compared. Stored in bank_fingerprints, scored in Python.
- A resemblance sets bank_source = "resemblance", is capped at 0.9 confidence, and declines to answer when two banks match about equally. Ambiguity produces no answer rather than a coin toss.
- Precedence: user > account > printed marks > bank code > resemblance.
- The correction response now reports what was learned (learned, learned_appearance) and the UI lists it, so "it remembered" is checkable.
- Migration e5b2c907da41. Verified: 127 tests pass on SQLite and Postgres 16; upgrade over existing rows leaves the new columns null (truthful — those documents were never examined for them) and downgrade is clean.
- 2026-08-05Bank corrections that stick
- The bank on a statement can be corrected: PATCH /api/statements/{id}/bank.
- A correction is remembered, not just applied. It is stored as BankHint rows keyed on signals derived from the account number (account_signals), so a later statement for the same account — or another account at the same institution — is identified from the correction instead of being guessed at again. See ADR-010 for why the account number and not the document's look.
- Statement.bank_source (parser | hint | user) records which route produced the name, and the UI shows it. A user's answer is never overwritten.
- Confirmed bank names are also passed into the parser prompt, which fixes the narrower "right bank, different spelling" case.
- New StatementService; statements are no longer read through the container directly from the route handler.
- Migration c3a1d84f2b07. Verified: upgrade over existing rows backfills bank_source to parser (truthful — they were parser guesses), and downgrade is clean. All 113 tests pass on SQLite and on Postgres 16.
- 2026-08-05ADR-010Bank corrections are learned, keyed on the account number
- ADR-006 has the model identify the bank from how a document looks — logo, colours, typography. That works well and is sometimes wrong in a way the model cannot detect: white-label and sub-brand banks are designed to look like their parent. The case that prompted this was a Moey statement identified as Wise.
- 2026-08-05ADR-011Recognising a bank without an account number
- ADR-010 keyed corrections on the account number, because it is the one thing on a statement that is not a judgement call. That was right, and it was too narrow. A photo of the top half of a page, or a statement whose footer was cropped, carries no account number — and the owner's point stands: a person looking at that page would still know the bank, from the colours, the typography, the layout, the language, and the name printed on it.
- 2026-08-04Deployment
- Added D9/ADR-009: Render now (set up from a phone), owner's own server soon.
- Docker images for both services; render.yaml and docker-compose.yml wire the same images up for the two hosts.
- Postgres is now the deployed database, SQLite only local development. Render's free tier has no persistent disk, so SQLite would be wiped on every redeploy.
- DATABASE_URL is normalised in config.py, because providers hand out postgres:// and SQLAlchemy 2 rejects it.
- Migrations run from docker-entrypoint.sh on every start.
- The test suite can run against Postgres via FINAID_TEST_DATABASE_URL. Verified: all 58 tests pass on Postgres 16 as well as SQLite.
- docs/deployment.md covers both hosts and the migration between them.
- 2026-08-04Initial build
- Established decisions D1–D8 with the project owner.
- Set up the ports-and-adapters backend skeleton with the four layering rules.
- Implemented phase 1 end to end: upload → parse → classify → review → chat.
- Implemented the Financial Agent (orchestrator/chat) and Expense Agent (tools over transactions), with a tool registry that later agents plug into.
- Built the responsive Next.js PWA: upload, review, dashboard, chat.
- Deliberately left unimplemented, with seams in place: Budget Agent, Stocks & Bonds Agent, manual movement entry, PSD2, broker integrations, real authentication.
- 2026-08-04ADR-001Python backend, TypeScript frontend
- The system needs document parsing, LLM orchestration, and two UIs. A single-language TypeScript monorepo would give shared types end to end; a Python backend gives a stronger data and document ecosystem.
- 2026-08-04ADR-002One responsive PWA instead of a separate native app
- The brief requires "a mobile UI plus laptop UI". That is either one responsive web app or two frontends.
- 2026-08-04ADR-003SQLite now, Postgres later, behind repositories
- The owner asked for the simplest thing now with an abstraction layer that makes a real multi-device, multi-user database easy later.
- 2026-08-04ADR-004Uploads only in phase 1; PSD2 behind a port
- Real PSD2 access requires a licensed aggregator (GoCardless Bank Account Data, Tink, TrueLayer), an account, credentials, and 90-day consent renewals. That is a meaningful amount of work before any software does something useful.
- 2026-08-04ADR-005Single user now, multi-user-ready schema
- The tool is for one person today, but retrofitting multi-tenancy touches every table and every query.
- 2026-08-04ADR-006Statement parsing via the LLM, with a per-bank parser seam
- Bank statements have no common format. Deterministic per-bank parsers are accurate but need writing per bank, and break when a bank changes its layout. The diagram also requires identifying the bank from visual cues (logo, colours, typography), which is inherently a vision task.
- 2026-08-04ADR-007Sub-agents are tool providers, not nested conversations
- The diagram shows a Financial Agent above an Expense Agent, a Budget Agent, and a Stocks & Bonds Agent. That could be implemented as nested LLM conversations or as one conversation with a partitioned tool surface.
- 2026-08-04ADR-009Render first, self-hosted second; Postgres from the start
- The owner needs the app usable from a phone now, and intends to move it to their own server within about a week. They cannot use a command line in the meantime.
- 2026-08-04ADR-008Money as signed integer minor units
- Floating-point money is a well-known source of silent corruption.
Generated from CLAUDE.md and docs/decisions.md. Nothing here is written twice.