Qiushi Engine posts BabyLM training results; preprint targets concept-level objectives

Qiushi Engine published a three-stage research program on arXiv cs.CL (arXiv:2609.10702) that describes how it built and then improved a language model on BabyLM 2026 Strict-Small inside a budget of 10 million corpus words and 100 million cumulative word presentations. The paper frames the effort as long-horizon, end-to-end autonomous research in which each model advance becomes the basis for improving how the next round of research is run. Its three stages are frontier advancement, principle discovery, and principle-guided model improvement, connected so that findings from one stage set the conditions for the next.

The stated constraint is that learning from limited text requires models to use context, generalize to new inputs, and retain useful capabilities. Stage I attacked the budget directly by combining compact restatements, budget reinvestment, and residual incremental learning to build the program's frontier model. Stage II turned to how repeated text is presented, and found that exact repetition and aligned restatement produce different patterns of context use, with the pattern depending on the target relations and the prediction window being tested. The result the paper emphasizes is negative: on controlled tasks, recovering familiar performance did not ensure that unseen inputs could still use the computations the model had learned.

Those findings were compressed into a testable principle. Data-efficient learning, the paper argues, should organize experience around the contextual dependencies needed for prediction, design visible information, supervision, and preservation as separate decisions, and test learning, generalization, and retention as separate outcomes. Stage III applied the principle by retaining source text and masking more local clues, but the abstract's description of its supervision cuts off mid-sentence, so the full design of the final stage is not visible in the posted text.

A second preprint in the same arXiv cs.CL listing, dated 11 September 2026, answers the data-and-compute constraint from a different direction. NCP-ArchPreview (arXiv:2609.10715) is described as a latent-space language model that pushes autoregressive pretraining past standard next-token prediction. It keeps token-level generation and adds Next Concept Prediction, an objective that asks the model to predict discrete concepts spanning multiple tokens. The report calls the concept objective explicit and more challenging than next-token prediction. The model builds its latent space by constructing a product-quantized concept vocabulary directly from its hidden states; the abstract breaks off at that point, before the training procedure is fully described.

The two reports sit at the same intersection from opposite ends. Qiushi Engine rearranges what the model sees and when it sees it, holding the objective fixed. The NCP-ArchPreview report sets the corpus question aside and changes what the model is asked to predict at each step. Both were posted as new arXiv cs.CL items, and both treat the volume of available text as the binding constraint on capability rather than something to be worked around with a larger corpus or more compute. Neither abstract reports outside replication, and the material available does not show whether either group has evaluated against the other's benchmark setting.

What the postings leave unresolved is largely in the truncation. Qiushi Engine's principle is explicitly framed as testable, which puts the next step outside its own program: independent work would need to reproduce the context-use and retention results under the same Strict-Small budget. The NCP-ArchPreview title signals a preview rather than a final architecture report, so the training details and evaluations that would let a reader judge the concept objective against standard pretraining are not in the posting. Until those documents surface, the claims from both rest on the authors' own abstracts.