By jankes72
21 October 2026
I am developing SSI MultiQuant as the next stage of SSI V5: bringing model size, memory use and measured task quality into the decisions made by an agent system.
One published comparison is deliberately narrow. An experimental Qwen2.5-Coder 1.5B GGUF file is 3.14% smaller than a standard Q4_K_M control. In a short CPU test, its sampled process peak RSS was about 1.82% lower, while elapsed time was effectively unchanged. Quality equivalence is still unresolved.
The larger goal is to connect this work with Universal Lab, Research Memory, Router V10, the planned S10 selection path and SCC execution. That integration remains under development. This article separates the measurements already published from the workflow I want to qualify next.
Why this belongs inside SSI
SSI V5 is being developed around a Director that plans work, assigns execution and collects evidence. Its model choices need to fit the task and the available hardware.
A configuration that saves memory on a short formatting task may behave differently on a longer programming task. A useful model profile therefore needs more than a file-size label: it needs a record of what was tested, on which hardware, with which constraints, and with what outcome.
MultiQuant is the experimental model-configuration layer in this work. I want SSI to compare complete configurations and retain the evidence behind each choice.
Universal Lab and Zero Lab
Universal Lab is the research collaboration environment around SSI. The design brings participant conversation, Director interaction, bounded experiments and inspectable session evidence into one workflow.
The published Universal Lab baseline includes authenticated web access and a configured translation bridge on modest development hardware. The later A16 installation report records 48 topics in Polish and English at three levels of detail, giving 288 prepared answer variants. Its build recorded 41 passing Python tests. Installation and HTTP readiness were reported by the operator; live narration remained a separate validation step.
Zero Lab is the separate experiment and validation layer. Its role is to freeze the protocol and evaluation criteria before execution, preserve the resulting evidence, and retain unsuccessful or inconclusive runs. Universal Lab provides the collaboration setting; Zero Lab supplies bounded experimental discipline.
This is where MultiQuant fits: a model configuration can become a candidate for a controlled experiment, with its memory savings assessed alongside the behavior that matters for a particular task.
What the model comparison measured
The published control report compares a source F16 artifact, a standard Q4_K_M artifact produced using llama.cpp, and an experimental SSI MultiQuant Q4KM artifact.
| Artifact | Whole GGUF file size in decimal MB |
|---|
| F16 reference | 3093.67 |
| Standard Q4_K_M | 986.05 |
| SSI MultiQuant Q4KM | 955.05 |
The SSI file is 30,994,944 bytes smaller than the standard Q4 control, a reduction of 3.14%. These are whole-file measurements. Standard Q4_K_M already uses different precisions for different tensor types; mixing precisions alone does not establish a novel method.
The short native CPU comparison used one CPU thread, a 1024-token context, a 16-token generation limit and a 6 GiB cgroup memory limit. The task was elementary arithmetic.
The four-run order was standard Q4, SSI, SSI, standard Q4. There were two runs per configuration.
| Mean measurement in this series | Standard Q4_K_M | SSI MultiQuant |
|---|
| Sampled process peak RSS in MiB | 1623.755 | 1594.165 |
| Total elapsed time in seconds | 3.7605 | 3.7610 |
| Correct short answers | 2 of 2 | 2 of 2 |
The sampled RSS difference was about 29.59 MiB, or 1.82%. RSS describes the resident memory of the sampled process, not the total memory consumption of the host.
The elapsed-time difference was 0.0005 seconds. Variation between successive runs was much larger, so this series supports no speed advantage. The arithmetic answers establish only that these short native runs completed correctly.
Quality remains part of the decision
Earlier R14 checks covered 24 cases:
| Group | F16 reference | SSI candidate |
|---|
| Python AST | 8 of 8 | 8 of 8 |
| Typed Routing | 5 of 8 | 5 of 8 |
| SCC Context | 4 of 8 | 3 of 8 |
| Total | 17 of 24 | 16 of 24 |
One SCC Context case passed for the reference and failed for SSI. The other 23 case verdicts agreed.
This comparison has material limits. Exact source-checkpoint parity was not established, the standard Q4 control was not evaluated on this quality suite, and generated code was not executed in R14. The additional failure cannot therefore be attributed solely to quantization.
The candidate remains SHADOW-HOLD: available for research evaluation, without production qualification. The practical result so far is a smaller file and a lower sampled RSS in a limited comparison, with more quality work required.
Large model feasibility under an 8 GiB limit
A separate R7 experiment has already tested complete public low-bit checkpoints, including Bonsai 2 27B, under an 8 GiB cgroup RAM limit. The Linux CPU host used four inference threads, no system swap, one request at a time and a 512-token context.
| Public checkpoint | File size in decimal GB | Peak process RSS in GiB | Simple checks passed | Observed short reply time |
|---|
| Bonsai 2 27B PQ2_0 | 7.21 | 7.13 | 3 of 3 | 8.9 to 83.3 seconds |
| Qwen2.5-Coder 32B Instruct Q2_K | 12.31 | 7.38 | 3 of 3 | 72.4 to 147.5 seconds |
| Llama 3.1 70B Instruct IQ2_XXS | 19.10 | 7.13 | 3 of 3 | 186.1 to 371.6 seconds |
All nine simple final checks passed, with no recorded out-of-memory event. The checks covered arithmetic, context extraction and structured sorting.
These runs used native inference and file-backed paging. A file larger than resident RAM can be read through the operating system as needed, at a substantial I/O and time cost. The result does not mean that all 70B weights fit in RAM at once, or that an SSI model was distilled from that checkpoint.
Bonsai was affected by a concurrent download, and the final 70B run used a different configuration from the smaller models. These timings are observations, not a controlled cross-model speed ranking. Three simple checks per model also do not establish general task quality or practical long-context performance.
My next goal for an 8 GB computer
I intend to prepare and validate an SSI-compatible model configuration for a computer with 8 GB of physical RAM, accepting longer response times where necessary. The aim is a useful, measured configuration that can participate in the wider SSI workflow.
An 8 GiB process-group limit on a research host is different from an entire computer with 8 GB available to its operating system and applications. The target configuration must be evaluated with that overhead, storage access, context length and realistic tasks included.
I will report task quality, peak memory, time to first output and complete response time, including unsuccessful cases. The model size and configuration should follow those measurements. The existing Bonsai, Qwen and Llama feasibility runs are starting evidence for that work, rather than a guarantee for every 8 GB machine.
What readers can get from GitHub
The public SSI V5 repository provides downloadable architecture documents, experiment reports, benchmark summaries, protocols and declared research limits. The published R1 to R7 memory report includes the Bonsai 27B feasibility result above; the separate MultiQuant control report records the Qwen 1.5B measurements.
Readers can use those materials to review the research progression and identify a bounded scenario they would like to discuss or test. Universal Lab is the wider collaboration and experiment environment described by those documents.
The public repository is the evidence and documentation layer. It does not currently provide a complete Universal Lab installer or redistribute the tested model weights. The proprietary SSI implementation and detailed model recipes remain private.
Connecting Research Memory with S10 and V10
The next development stage is the connection between stored experimental experience and the tasks SSI chooses to run.
The proposed division of responsibility is:
| Component | Intended role in the integrated workflow |
|---|
| Director | Generate bounded tasks and assign work against the mission and resource budget |
| Research Memory | Retrieve relevant prior outcomes and retain the context of each new result |
| S10 selection path | Help select a qualified micronetwork or specialist capability |
| Router V10 | Select a verified model and resource profile for the task |
| SCC execution path | Carry out bounded work and return observable results |
| Zero Lab | Fix the experimental contract and evaluate against its declared criteria |
S10 is kept distinct from SCC in this roadmap. These names identify different proposed responsibilities; they are not interchangeable labels.
A full production connection between these components and MultiQuant has not yet been established by the published evidence. The R9 laboratory router work is separate from production Router V10: the public report records eight checked plans, all retained in HOLD.
How the next tasks should be generated
In the intended loop, Research Memory supplies relevant earlier cases, measurements and unresolved questions. The Director turns those into bounded tasks with an objective, resource limits and evaluation criteria. S10 and V10 then help choose the specialist path and model configuration.
After execution, Research Memory should retain the actual outcome together with the task context, model profile, hardware constraints, timings and evidence reference. The next selection can then use that experience.
For example, the R14 SCC Context disagreement gives us a concrete follow-up. First establish common checkpoint and tokenizer provenance. Then compare F16, standard Q4 and the SSI candidate on a representative held-out task family, while preserving the original case as a diagnostic.
The disputed example must not become the only target for a repair. Any adaptation needs separate training or calibration material and a held-out evaluation, so that learning the example does not masquerade as broader improvement.
This is the purpose of generating tasks from experience: each experiment should resolve a specific uncertainty and produce evidence useful to the next decision.
Calibration and adapter experiments
The planned AutoRepair work broadens the choices under evaluation. Calibration and adapter-based approaches such as LoRA should be compared in context rather than assumed to have a universal ranking.
Further investigation includes QLoRA, LoftQ and adapter composition or selection, including X-LoRA. These remain research directions for this integration. No improvement to the measured Qwen candidate is claimed here from those methods, and no completed distillation is claimed.
The object of comparison should be the complete configuration: model, quantization, optional adaptation, task family and hardware budget. An untested variant should remain untested in Research Memory rather than receive an invented score.
What comes next
The immediate measurement priorities are common checkpoint and tokenizer provenance, a standard-Q4 control on the same representative quality suite, and more repetitions with varied run order. The measurements should distinguish model loading, prefill, decoding, process memory and cgroup memory.
Integration qualification is a separate step. It needs to show that the Director can generate a bounded task, the intended selectors can make an evidence-based choice, execution can return a verifiable result, and Research Memory can preserve the experience without changing the original verdict.
That is the next stage of SSI development I am working toward: model optimization becomes one measured input to an agent system that can choose, test and improve its configurations.
Public evidence
The public material shares architecture, measurements and their limits. Proprietary implementation, model recipes, private prompts, weights and operational data remain private.
AI assistance was used to prepare and edit the English text.
