LeJEPA · Principled Training Cleanup · SIGReg and Isotropic Embeddings
Abstract LeJEPA (2025) introduces a cleaner, more principled JEPA training objective (SIGReg), argues for isotropic embedding structure, removes heuristics, and emphasizes scalability. It is the theory-and-training cleanup layer. The structural gaps persist: simplifying the objective does not introduce the missing symbols.
6 FORMAL GAPS · 1 PER CANON SYMBOL
SIGReg Objective Has No Formal Invariant
γ₁ — THE FLOOR
LeJEPA's SIGReg (Spectral Isotropic Gradient Regularization) objective encourages isotropic embedding distributions. Isotropy is a distributional property — it describes the shape of the embedding cloud — but it is not a formal invariant in the γ₁ sense. An isotropic embedding has no fixed grounding point. The floor is absent even in the cleaned-up version.
Isotropic Embedding Structure Not Formally Self-Adjoint
H=H† — THE HONEST GATE
LeJEPA argues that isotropic embeddings are more principled than collapsed or directional ones. But isotropy does not imply self-adjointness. A self-adjoint encoder satisfies H=H† — encode(x) is verifiable against decode(encode(x)) in a symmetric way. Isotropy is a statistical property. Self-adjointness is an operator property. LeJEPA conflates the two.
No Paradigm Audit Across Simplification Choices
LSOS — THE READER
LeJEPA removes multiple heuristics from JEPA training. Each removal is a paradigm choice. There is no formal audit of when a simplification changes the learned paradigm vs when it preserves it. LSOS would read the active training paradigm and flag when a simplification has introduced an unacknowledged shift.
No Reset When Simplified Objective Collapses
WLD — THE RESET
When LeJEPA's simplified SIGReg objective leads to training collapse — despite the cleaner formulation — there is no mercy reset. The simplification reduces the number of heuristic safeguards, making collapse events more likely when the objective alone is insufficient. WLD provides a principled collapse recovery that LeJEPA's design does not include.
No Continuity Guarantee After Heuristic Removal
FEP — THE SWITCH
LeJEPA removes augmentation heuristics and training tricks present in earlier JEPA models. There is no formal guarantee that representations learned without these heuristics are continuous with those learned with them. The paradigm switch from heuristic-JEPA to principled-LeJEPA may produce incompatible representations.
Scalability Ceiling of Minimal Objective Undefined
FOF — THE BREACH
LeJEPA claims that its minimal objective scales well. The upper bound of this scalability claim is not formally defined. At what model size or data scale does the SIGReg objective become insufficient? This ceiling is not named. FOF names the ungovernable boundary of the minimalist approach.
STE COMPLETION LAYER
What changes when you add the 8-symbol Canon
Adding the Canon to LeJEPA does not change the architecture. It adds the missing structural layer:
⚓ γ₁ — invariant anchor: mathematical ground truth latent representations must converge to.
⯛ H=H† — honest gate: bidirectional verification of every prediction.
〰️ LSOS — paradigm reader: reads active paradigm before reasoning begins.
🌀 WLD — mercy reset: detects collapse and resets to last stable state.
γ FEP — safe switch: continuity guarantee across paradigm transitions.
🌌 FOF — named ceiling: formal boundary of what the architecture can claim.
═ EVEN — substrate: ground beneath all the above. What holds when everything else is active.
The Canon is not an add-on. It is the formal completion of the JEPA programme.
X POST · @ylecun
POST 1 — Name the gap
@ylecun LeJEPA (2025): Gap 2 (H=H†) — predictor not self-adjoint. Asymmetric by design. Not an empirical limitation — a missing symbol. pemos.ca/lejepa-gap
POST 2 — Canon map
@ylecun LeJEPA: 6 gaps · γ₁ (no anchor) · H=H† (no gate) · LSOS (no audit) · WLD (no reset) · FEP (no continuity) · FOF (no ceiling). Same in all 14 milestones. pemos.ca/jepa-index
POST 3 — Invitation
@ylecun LeJEPA gap analysis: part of a 14-milestone series. Same 6 structural gaps in every milestone. The gaps are there because the symbols were never in scope. They are now. pemos.ca/jepa-index
LeJEPA is a landmark in the JEPA lineage. The 6 gaps we identify are not critiques of the engineering — they are structural absences that the Canon fills. Each gap maps to a symbol that was always going to be necessary once the JEPA architecture matured. The Canon did not wait for the JEPA timeline; the JEPA timeline arrived at the Canon. The gaps are there because the symbols were never in scope. They are now.
Gap 1 (γ₁): SIGReg Objective Has No Formal Invariant LeJEPA's SIGReg (Spectral Isotropic Gradient Regularization) objective encourages isotropic embedding distributions. Isotropy is a distributional property — it describes the shape of the embedding clo...
Gap 2 (H=H†): Isotropic Embedding Structure Not Formally Self-Adjoint LeJEPA argues that isotropic embeddings are more principled than collapsed or directional ones. But isotropy does not imply self-adjointness. A self-adjoint encoder satisfies H=H† — encode(x) is verif...
Gap 3 (LSOS): No Paradigm Audit Across Simplification Choices LeJEPA removes multiple heuristics from JEPA training. Each removal is a paradigm choice. There is no formal audit of when a simplification changes the learned paradigm vs when it preserves it. LSOS w...
Gap 4 (WLD): No Reset When Simplified Objective Collapses When LeJEPA's simplified SIGReg objective leads to training collapse — despite the cleaner formulation — there is no mercy reset. The simplification reduces the number of heuristic safeguards, making ...
Gap 5 (FEP): No Continuity Guarantee After Heuristic Removal LeJEPA removes augmentation heuristics and training tricks present in earlier JEPA models. There is no formal guarantee that representations learned without these heuristics are continuous with those ...
Gap 6 (FOF): Scalability Ceiling of Minimal Objective Undefined LeJEPA claims that its minimal objective scales well. The upper bound of this scalability claim is not formally defined. At what model size or data scale does the SIGReg objective become insufficient?...
The STE provides the completion layer for each gap. The gaps are not empirical — they are structural. Adding the symbols closes the gaps by definition.
Here's how to explain LeJEPA gaps to a 10-year-old:
Gap 1 — No floor: Imagine you're building a tower and you don't have a foundation. The tower might stand for a while but there's nothing it's guaranteed to come back to if it leans. γ₁ is the foundation. The math already built it. LeJEPA didn't use it.
Gap 2 — No honest check: If I tell you something, and you can't ask me 'are you sure?' and get the same answer backwards, I might be guessing. H=H† is the rule that every claim must survive being asked backwards. LeJEPA makes claims it can't verify in reverse.
Gap 3 — No paradigm reader: Sometimes you're playing one game and someone changes the rules without telling you. LSOS reads the rules before you play. LeJEPA changes paradigms without reading them first.
Gap 4 — No reset button: When you're drawing and you make a mistake, you have an eraser. WLD is the eraser. LeJEPA doesn't have one. If it goes wrong, it keeps going wrong.
Gap 5 — No safe switch: If you switch from reading a book to watching a movie, you expect to pick up where you left off. FEP is the bookmark. LeJEPA switches paradigms and might drop the book.
Gap 6 — No ceiling: Every ladder needs to know when to stop. A ladder that claims it goes forever is a lie. FOF names where the ladder ends. LeJEPA hasn't named its ceiling yet.
The Canon is not a critique. It's the toolkit that was always going to be needed. LeJEPA got as far as it could without it. Now the toolkit exists.
The 6 gaps identified in LeJEPA are structural absences in a published architecture. Identifying structural absences in published work is not IP infringement — it is prior art analysis. The STE symbols that fill these gaps are EOSE IP. The gap identification is public analysis. The completion layer is proprietary.
Status: LSOS-OWNERSHIP-001 open (EOSE Labs Inc. not yet registered). No public disclosure of STE completion layer until registration clears. Gap analysis pages (like this one) are disclosure of the gaps, not the fills. This is legally distinct and permissible.