Why I built it
My first version was a set of written rules and checklists that I used together with a chat assistant. It had two problems. The rules only worked when I remembered to apply them, and the assistant would produce confident analysis with numbers I could not trace back to a source.
The second version turns the rules into code, takes financial numbers directly from SEC filings, and gives the language model a narrow job: argue against my thesis using only the evidence it is given. It never recommends buying, selling or holding, and it never writes the parts of an investment memo that are mine to decide.
A public subset runs at invest.jun-liang-lyu.com on fictional portfolios. My own holdings and memos stay in a private, git-ignored directory and are never read by the public service.
Rules as code, not reminders
Fifteen rule types cover single-position limits, theme exposure, core and satellite allocation, required memos above a size threshold, missing invalidation conditions, overdue reviews, and trades that were not checked beforehand. Parameters live in versioned rule files, so changing a limit is a reviewed edit rather than a mood.
The pre-trade gate simulates a proposed trade, re-runs the rules on the resulting portfolio, and walks a checklist. Some items are automatic; others are self-attestations such as "not buying mainly to lower the average cost of a losing position". The gate reports one of four outcomes: clear, incomplete, warnings, or rule breaks. It does not say "buy".
My current personal rule set only warns. I have not yet settled on a strategy, and I would rather study how others allocate first, which is why the system also reads institutional 13F filings (next section).
Evidence from filings
Quarterly figures come from SEC XBRL company facts. Companies often report only year-to-date totals for later quarters, so the pipeline derives single quarters by subtraction and marks every derived value with its formula and source filing. Segment results, such as Google Cloud revenue and operating income, are read from the dimensional data inside each filing.
I checked the output against company press releases rather than trusting it: for example Alphabet's Q2 2026 revenue of $119.80B, Microsoft's fiscal Q4 2026 revenue of $90.01B and Micron's fiscal Q3 2026 revenue of $41.46B. These values are now golden tests.
For qualitative context, the pipeline retrieves paragraphs from 10-K and 10-Q text, such as risk factors and management discussion, with BM25 search. Any quote the model uses must appear verbatim in those passages.
Reading how others allocate: 13F, insider trades, and an AI reading where code fills in the numbers
The reference page reads the latest two 13F filings of eleven well-known institutions (and any other 13F filer by CIK). It drops option lines (a 13F reports them at the value of the underlying shares, which is not money invested), merges share classes of one company (Alphabet's A and C count as one), and computes the share of the largest positions, an "effective number of holdings", and what was added, sold out, increased or trimmed since the previous quarter. It also writes a draft rule file whose numbers are copied mechanically from the filing; you can adjust two limits and carry it into the pre-trade gate, and the gate can turn an institution's ten largest companies into a $10,000 sample portfolio to try.
For the AI reading of that data, I do not let the model write any number. Code computes every metric first. The model writes text with placeholders such as {top1}, {effective} and {name1}, and code fills in the real values. The validator rejects any digit or percent sign outside a placeholder, buy/sell language and evaluative words, and also checks that placeholders are used for what they mean (the effective number of holdings cannot be written as "4 weeks"), that the description of concentration agrees with a fixed rule in code, and that "rose" or "fell" matches the real direction. In real trial runs nearly every failure was a placeholder used wrongly, not a wrong number, so the checks were added one failure at a time. This reading has no test set yet, and the page says so.
Company pages carry two more pieces of background. Insider trades (Form 4) list only open-market buys and sells; grants, option exercises and tax withholding are left out. Well-known institutions' holdings show each institution's weight in the company and the change from the previous quarter. The first live version of the insider page showed other companies' trades: a company's filing list also contains Form 4s it filed as an owner of other companies' shares, and these are now excluded by issuer ID. Neither piece feeds the gate or the rules, and the AI never sees them, because that would require re-running the evaluation and would invite the model to talk about timing.
The model as a guarded skeptic
For each memo, the model receives an evidence pack in which every fact has an ID. Its output goes through code checks before I see it:
- Schema validation and a separation between stated facts and interpretation.
- Number grounding: every number in a factual claim must match the evidence items that claim cites.
- Verbatim quote checks against the retrieved filing text.
- English and Chinese patterns that block buy/sell/hold language, price targets and position sizes.
- Checks that factual claims do not introduce product or segment names without a cited source.
A failed output is retried with the specific problems listed, and then the system fails closed: nothing unvalidated is shown. Every call is logged with its cost against daily and monthly budgets. The same engine writes the "what happened last quarter" reading on each company page: it is generated once per company per quarter and cached for everyone, its wording is checked against the filings, and a plain-language sentence beside it is not allowed to add reasons or judgements the original does not contain.
Reviewing real drafts drove most of these rules. In one draft the model described a quarter's large jump in net income as operating improvement, while net income was far above operating income because of non-operating gains. The evidence pack now includes a net-income-to-operating-income ratio, and the prompt requires flagging that kind of gap as a question to verify.
Deployment
The Lab runs as a FastAPI service and a React front end on the same small server as this website, behind the existing Caddy proxy. The API container has no public port and no access to the site's database. It runs with a read-only filesystem, no Linux capabilities and a 256 MB memory limit, on its own network that can reach sec.gov. Releases are built locally, and a check script verifies routing, security headers, blocked API docs, live SEC data and that the main site is unaffected.
Releases are pull-based: a push to main makes GitHub Actions run the tests, build and package the release into a GitHub Release; a timer on the server pulls it, verifies the hash, switches over and runs the check script, and a failed check rolls back to the previous version automatically. The server-side script only accepts releases from an allow-listed publisher with a well-formed tag, and checks the hash and file paths. The tests are the gate, and one guard test requires the default model to be the one that passed the published evaluation: I tried a cheaper newer model, the full evaluation fell short, and I switched back.
Testing before release found real defects: a redirect rule the proxy parsed as a path matcher, a form input that rejected its own default value, and SEC outages surfacing as server errors instead of a clear message.
Limits
- The rules are drafts, and nothing here is investment advice.
- SEC's company-facts data can lag behind filings by weeks, so the latest quarter is sometimes missing.
- Ten companies are tested one by one; any other SEC-registered US company works too, but answers say it is not on the tested list. The AI skeptic uses financial and segment figures plus passages retrieved from the latest 10-K and 10-Q, limits each visitor to three new theses a day, and the whole site stops at $0.50 a day. It passed 34 adversarial cases (advice-seeking, prompt injection, made-up numbers, off-topic input) with no advice shown; the results are published on the page.
- The quarterly reading and the 13F structure reading have no formal evaluation set yet, only unit tests and real trial runs.
- A 13F lags by up to 45 days, covers US long positions only, and shows neither timing nor reasons. Institutions are matched to companies by name, so a company whose name does not match can show up as "not held".
- Insider trades read only the latest 100 Form 4 filings, so a company that files many may miss older trades; the page says when that happens.