JigLoom — AI & software product studio

DeepSeek-V4.1-Flash

Causal encoder–decoder · CSA2 · DeepSeekMoE

A 20-layer causal encoder supplies hidden states to a 20-layer decoder. Cache reuse and sparse indexing reduce repeated work.

Text + native vision

DeepSeek-ViT features and text embeddings enter Single-Pass mHC. Engram is conditional token memory, not document retrieval.

Causal encoder–decoder

The encoder’s final hidden states supply the decoder’s global KV projections. Each half contains 20 layers.

Sliding-window attention

The first two encoder layers use a local window. Deeper CSA2 layers also combine selected main KV with layer-local SWA KV.

Hierarchical sparse indexer

Full scores all causal positions. Selected blocks form a shared candidate pool. Reindex searches only that pool; Reuse retains the last selected indices.

384 routed · 6 active · 1 shared

A router selects six expert paths per token, while a shared expert contributes alongside them. Weighted outputs recombine.

DSpark

A separately trained semi-autoregressive drafter and confidence-scheduled verification accelerate decoding; this is not an MTP training branch.

Production implication. Cache generation, indexing and expert execution are different costs; design the serving system around each.

Kimi K3

KDA + gated MLA · Block AttnRes · Stable LatentMoE

A 93-layer hybrid: 69 KDA layers and 24 gated MLA layers. Block Attention Residuals mix representations across depth.

MoonViT-V2 + text

Native visual features are projected into the shared language embedding space.

Kimi Delta Attention

Short convolutions create Q, K and V. Channel-wise retention α and write strength β update a recurrent state; Q reads that state.

Gated multi-head latent attention

A compressed KV representation supplies key/value projections. A separate sigmoid gate modulates the attention output.

Attention across depth

Learned pseudo-queries weight embedding, previous block summaries and the current partial sum. These are depth sources, not token positions.

896 routed · 16 active · 2 shared

Routed experts operate at latent width 3584, with full-width shared paths. RMSNorm and SiTU-GLU stabilize the routed path. Layer 1 uses a dense FFN.

Production implication. Recurrent state, latent KV and depth summaries have distinct lifetimes in a production inference engine.

Llama 4 Maverick

Native multimodality · early fusion · alternating dense / MoE

Image features and text tokens join one sequence before shared model processing. Dense and sparse expert layers alternate.

Text + image patches

A native vision encoder supplies visual representations alongside embedded text.

Early fusion

Projected image features and text embeddings join one shared sequence before the language backbone.

Alternating dense / MoE

Dense computation alternates with sparse MoE layers. The eight visible layers are a representative excerpt.

128 routed · 1 active · 1 shared

A single routed expert is selected for each token, alongside the shared expert.

Shared multimodal output

One decoder backbone produces an output conditioned on the fused context; there is no separate language encoder tower.

Production implication. Visual preprocessing, shared context and sparse routing belong to one multimodal product pipeline.

From architecture. To product.

JigLoom · engineered as one system

Model capability becomes useful through context, controls, product design and continuous evaluation.

Data

Validated inputs and governed access.

Foundation model

Select capability for the actual task.

Adaptation

Fine-tune or adapt where appropriate.

Retrieval

Retrieve relevant evidence.

Memory

Retain scoped application state.

Agent logic

Coordinate bounded decisions.

Tools / APIs

Execute authorized actions.

Product interface

Make the system usable.

Inference infrastructure

Serve reliably within a cost budget.

Observability

Trace latency, errors and behavior.

Evaluation

Measure quality before and after changes.

Feedback

Feed reviewed outcomes into improvement.

Production implication. This is an illustrative application architecture, not a claim about a deployed client system.

Generalized application architecture.