Changelog#

All notable changes to this project are documented here.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning. Versions are tag-driven (hatch-vcs); see the dated release entries below for published versions.

[Unreleased]#

Added#

  • A fixed-release, clean-room He–Kelly–Manela paper-data loader for the official 1970Q1–2012Q4 quarterly/monthly factors and 124 cross-asset test portfolios. It supports memory-only official download or a byte-identical explicit local path, pins archive SHA-256, validates the exact seven-member ZIP plus every CSV schema/calendar, and never executes or persists the bundled Julia code. Canonical frames expose decimal factors, same-period-RF-subtracted excess returns, class metadata, and class-local complete cases while preserving the combined unbalanced panel. The redundant All_01–All_124 block must duplicate the named assets cell-for-cell. Updated factor-only files are validated but not exposed. Path-free provenance records the author’s non-commercial/as-is terms, third-party-source caveat, and redistributable=false; build exclusions and distribution canaries reject ZIP, CSV, and Julia carriers.

  • A strict path-only, never-download loader for a caller-obtained Daniel–Moskowitz momentum archive. It validates the exact documented 12-member regular-file allowlist entirely in memory, rejects paths/links/duplicates and unsafe size/compression/text-object envelopes, and requires a frozen caller contract for the otherwise undocumented text delimiter/header/columns/date format/units plus optional SHA/date/row pins. Paired canonical daily/monthly frames expose ten decimal-return deciles and wml = decile_10 - decile_1; path-free provenance binds actual archive, member, paired-member, all-member-manifest, and parser-contract hashes and never asserts paper-exact identity on an incompletely pinned contract. Distribution exclusions and audit canaries reject all .tar.gz carriers and every documented raw member basename. Documentation makes explicit that the archive contains no separate monthly market-total, daily market-excess, or risk-free inputs required by the dynamic strategy. Secure source-path traversal requires POSIX dir_fd and O_NOFOLLOW; unsupported platforms fail closed before reading.

  • Strict, memory-only loaders for the frozen 2025-02-16 Global-q HXZ factor release. The q4 loader validates the complete q5 carrier before selecting the original R_MKT, R_ME, R_IA, and R_ROE columns, explicitly excludes R_EG, and returns month-end decimal returns with RF. The companion loader requires all 18 size/investment/profitability cells in every month and retains positive stock counts plus value-weighted total and ex-dividend returns. Both pin official URL, byte/line count, SHA-256, full 1967-01–2024-12 calendar, schema, finite values, and percent-to- decimal conversion before applying a date slice; redirects stay on HTTPS global-q.org, source bytes remain in memory, and path-free provenance marks the files non-redistributable.

  • A method-neutral, partitioned daily-WRDS equity collection for security-level reproductions. It caches 64 stable permno buckets by default through the existing immutable raw-zone protocol, keeps CRSP DSI value-weighted market and daily Fama–French RF as separate inputs, publishes only a complete canonical path-free manifest, and exposes iterator-only raw/clean consumption. Legacy SIZ is the primary convention and CIZ is an explicitly distinct sensitivity. Daily clean steps retain valid ret == -1, preserve missing observations, subtract only same-date decimal RF, and reject calendar/unit drift rather than filling. The existing monthly CRSP cleaner now accepts an explicit no-exchange-screen convention and performance_delist_fill=None when a reproduction must preserve an unreported return. A separate SIZ monthly target collection streams independent MSF/name-history/delist caches per bucket plus month-end-aligned decimal FF RF, retaining raw and adjusted returns so delisting identities and terminal-only rows stay auditable. Collection manifests and data_vintage bind the registered clean pipeline’s step versions and explicit parameters. Both long-running collections bind an immutable same-vintage CRSP release marker and compare it with WRDS before any new bucket and after all pulls, preventing an interrupted run from mixing releases. The paper-facing monthly iterator is fixed, preventing a sensitivity from silently reusing the primary recipe identity.

  • A strict path-only loader for a caller-obtained AQR Betting Against Beta Original workbook. It never downloads, discovers, copies, or caches workbook bytes; validates the Office envelope, exact sheets/header, formula/active-content policy, exact official paper hyperlink, YYYYMM dates, full U.S. release range, and percent display formats; preserves the already-decimal cell values; and attaches path-free hash/terms/unit provenance. Spreadsheet files are explicitly excluded from and rejected by distribution audits.

  • Strict, memory-only loaders for the paired official Pastor–Stambaugh aggregate-liquidity and historical-beta-decile files. They validate the author envelope, date range, monthly continuity, sentinels, decimal-return units, and supplied 10-minus-1 arithmetic; attach the Booth URL, allowlisted HTTP metadata, retrieval timestamp, raw-byte SHA-256, parser/unit recipe, and audited slice; and mark the revision-prone releases as latest-snapshot/non-PIT. The aggregate frame keeps raw innov_liq and adds the explicit liquidity_innovation = innov_liq / 100 reproduction contract. Documentation distinguishes the downloaded historical-beta portfolios from the predicted-beta portfolios required for paper Table 6 and requires RF subtraction before treating decile total returns as test-asset excess returns. No downloaded bytes are persisted or packaged.

  • Public Ken French multi-asset loaders for the 10 Industry and 25 Size–Book-to-Market panels and the momentum (Mom/WML) factor, all monthly or daily. load_ff_momentum names the factor surface explicitly; the existing portfolio_set="momentum" path delegates to it. Monthly Mom remains on the primary tidyfinance backend. Because tidyfinance’s daily Mom path currently returns no observations, daily Mom uses an in-memory, strict parser for the official Ken French CSV zip and records its URL, HTTP source timestamp, content SHA-256, and parse recipe. No raw download is persisted. frame_provenance / frame_data_vintage expose that metadata, and the view bridge preserves it.

  • A method-neutral compustat_annual_formation_clean step that preserves raw gross-profit inputs and makes records ending in calendar year t eligible at the June t+1 formation date. It deterministically resolves multiple fiscal ends within a firm/calendar year and rejects conflicting duplicate selected records.

  • The numeraire view factories (to_timeseries_view, to_multiasset_view, and to_cross_section_view) now accept a return_type passthrough ("simple" by default, or "log") that declares the algebra of the return column to numeraire. Because clean.py emits source_log_return-style columns, passing one as the return without this declaration would silently mix log and simple algebra downstream. The parameter is feature-detected: "simple" is never forwarded (so the factories still work against numeraire 0.2.x), and any non-simple value requires the numeraire >= 0.3 constructor — otherwise a clear “requires numeraire >= 0.3” error is raised rather than the declaration being dropped. The [numeraire] extra pin is intentionally left at >=0.2,<0.3; it is staged to move once numeraire 0.3.0 ships.

Changed#

  • load_ff_factors now validates the exact FF3/RF schema, rejects missing/non-finite values, moves monthly source labels to realized-return month-end, widens the backend query to boundary months before reapplying exact inclusive bounds to those canonical timestamps, and attaches requested/ backend/selected bounds plus honest tidyfinance version/recipe provenance (without inventing a raw-byte digest).

  • Breaking pre-1.0 provenance migration: recipe_hash(..., input_hashes=...) now requires a {role: hash} mapping instead of the legacy bare hash list, and DataLock.add_clean(..., inputs=) likewise requires {role: artifact_label}. Built now carries canonical recipe metadata and registered step identity and should be obtained from run_step, not constructed directly. Lock format version 1 is not reinterpreted as version 2; rebuild clean artifacts into a new lock. This change must ship in the next minor release rather than a patch release.

Fixed#

  • Annual Compustat investment growth (compustat_annual_clean) now requires the prior observation to be the immediately preceding fiscal year (a datadate exactly twelve months earlier) before it reports a year-over-year asset growth. Previously the growth was computed between row-adjacent records, so a firm history that skipped a year (or changed its fiscal-year-end mid-history) had its multi-period jump mislabeled as a single year of growth. Such non-consecutive gaps now yield missing inv; the docstring no longer overstates that row-adjacency is year-adjacency. The clean-step version is bumped so prior cached recipes are not mistaken for the corrected output.

  • clear_data_home is now contained: it resolves its target (rejecting a symlinked data home) and deletes only a real directory whose top-level entries are all expected numeraire cache members (raw, clean, fredmd, data.lock.json). A non-existent path is a no-op; a filesystem root, the user’s home directory, a plain file, or any directory holding unrelated entries is refused instead of being recursively removed. This mirrors the traversal hardening already applied to the raw-zone cache paths, so a mistyped or misconfigured $NUMERAIRE_DATA cannot destroy an arbitrary caller-supplied location.

  • Corrected the to_cross_section_view docstring: numeraire views now expose a provenance mapping, so the note that “the view itself holds no provenance state” was stale. The engine-level data_vintage stamp remains the way this bridge pins a result to its build.

  • Raw-cache readers now reject final parquet or metadata sidecars that are symlinks or non-regular files. Cache-presence checks treat them as invalid, and incomplete-cache recovery leaves such paths for manual inspection instead of following or unlinking a redirected artifact.

  • Annual Compustat investment growth now preserves missing total assets instead of implicitly forward-filling them. Growth is computed on the complete firm history before book-equity filtering, so even a missing-asset row later removed from output leaves both adjacent growth rates missing. The clean-step version is bumped so prior cached recipes cannot be mistaken for the corrected output.

  • CRSP cleaning now retains delist-only terminal months through an outer stock/delist merge, exposes is_terminal plus the applied delisting fields, preserves optional permco/siccd, and leaves market capitalization missing when the terminal row has no price observation.

  • Corrected the annual Compustat documentation: datadate + report_lag_months is a fixed-lag heuristic retained for compatibility, not the Fama–French following-June formation rule.

  • Raw WRDS cache reads now verify the sidecar identity, query hash, row count, and content digest before reusing a same-source/same-vintage parquet. Query hashes and the sidecar’s actual PIT status are chained into data.lock.json and the downstream clean recipe hash, together with the content-digest contract version; relabeling identical bytes from snapshot to vintage therefore moves the clean data_vintage. Callers must choose a new vintage because an existing raw cache is never overwritten in place. Query identity hashes exact SQL text; generic whitespace normalization is intentionally avoided because it can collapse distinct string literals.

  • Raw cache publication now stages parquet and metadata together and atomically renames the complete directory into place. A failed first write cannot poison an immutable vintage, and a legacy directory containing only one of the two known artifacts is safely rebuilt. Every read surface, including read_raw, pull_raw, and pull_standard, uses the same validation path. An OS-released writer lock serializes concurrent first pulls, avoiding duplicate licensed SQL and full-frame memory/disk work; a subsequent pull removes abandoned staging data after a crash. Dedicated lock/staging namespaces reject symlinked components before creation or cleanup, so an internal cache link cannot redirect file operations outside the configured data home.

  • Frame content digests now pin semantic schema as well as values: extension dtypes, categorical levels/order, timezone, object-scalar type/missing-sentinel identity, and column/index metadata participate in the hash. Raw sidecars and raw/clean lock entries record content-digest contract version 2; caches from the older value-only contract must use a new vintage rather than being silently reinterpreted.

  • Raw cache source and vintage labels must be single path components, preventing a malformed label from escaping the configured data home during cache publication or incomplete-cache cleanup; @ and # are reserved for unambiguous lock and chaining identities.

  • Source distributions explicitly exclude local Sphinx build and generated-API output, and CI inspects archive members so generated HTML/doctrees, cache files, parquet, and raw metadata cannot be packaged. The release workflow runs the same shared archive audit, with exclusion canaries, before upload.

  • Recipe-hash contract version 2 binds each input role to its upstream hash, preventing exchanged inputs to a non-commutative step from producing the same provenance stamp. run_step now falls back to each missing input’s semantic content digest instead of its name, so default and partial input_hashes mappings remain sensitive to the actual data. Parameters use a recursive, type-tagged canonical form and are frozen before the step runs, distinguishing values such as a list/tuple or timestamp/string while rejecting opaque state instead of silently stringifying it. Nested mapping order is retained, custom containers and pandas scalar subclasses fail closed, custom NumPy scalars and opaque timezone objects fail closed, and registered steps must name parameters explicitly rather than accept variadic **kwargs. Unknown frame and input-hash roles now fail closed, preventing a misspelled pinned role from silently taking the digest fallback. Existing clean recipe hashes move intentionally. Lock format version 2 records canonical explicit parameter overrides plus each role’s artifact label and chained hash; defaults remain defined by the recorded step name/version in pinned package code.

  • Clean-lock insertion revalidates a built frame’s semantic content digest, rejecting outputs or digest metadata mutated after run_step. CRSP delisting returns are normalized with explicit numeric conversion, as are optional firm/industry identifiers before name-history filling, avoiding pandas’ deprecated object-downcast behavior for valid empty or object-typed pulls.

[0.1.1] - 2026-07-07#

Documentation refresh only — no functional changes.

Added#

Changed#

  • Refreshed the README with badges, an ecosystem cross-link, and a citation pointer; corrected stale “numeraire is not on PyPI yet” comments in packaging metadata.

[0.1.0] - 2026-07-06#

Added#

  • sources — tidyfinance-backed loaders. Thin loaders over tidyfinance for standard public sources (Fama-French factors, Goyal-Welch predictors, and the FRED / JKP / q-factor / OSAP surface it covers). Frame loaders (load_ff_factors, load_goyal_welch) return tidy frames and stay numeraire-free; the *_view helpers (load_gw_view) add an optional numeraire bridge and a data_vintage provenance stamp. This frame-vs-view split is intentional — see the README note.

  • Raw → clean → view data zones. A step registry with recipe-hash chaining (register_step / run_step / recipe_hash), deterministic clean builders for the standard WRDS-schema panels (crsp_monthly_clean, compustat_annual_clean, compustat_quarterly_clean, ccm_links_clean), a DataLock from which a numeraire result’s data_vintage is derived, and a zones.view bridge (to_cross_section_view) that lifts a clean panel into a numeraire CrossSectionView. The clean builders are pure transforms and run without credentials; only zones.wrds performs live, credentialed pulls.

  • builders/fredmd — vintage-aware FRED-MD. Self-built ETL for what tidyfinance does not cover: FRED-MD monthly vintages (reference period x vintage x series) with McCracken-Ng tcodes applied at build time (transform=True), plus vintage archive download/cache helpers.

  • WRDS zone (zones.wrds), optional [wrds] extra. Live CRSP / Compustat pulls into the raw zone from environment-only credentials (nothing written to disk), each cached as parquet with a _meta.json query-hash sidecar in the outside-repo data home — licensed bytes never enter git. Deselected in CI (network + wrds markers); the module imports fine without the extra.

  • Packaging — BSD-3 license, PEP 561 py.typed, per-version classifiers, and CI (lint, format, basedpyright, pytest, build) plus an inert Trusted-Publishing release workflow.

Notes#

  • Ships code, not data. No licensed or redistributable data is bundled; build outputs land in a local, git-ignored cache. Public/synthetic fixtures and mocked pulls only in tests.