Few details about the model have been made public, as far as I know.
However, Steano Ermon, one of the founders of Inception Labs, the company that released the model, co-authored a paper in late 2023 (revised in 2024), titled "Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution." The authors tested their proposed method on small autoregressive language modeling tasks, and claimed promising results:
However, Steano Ermon, one of the founders of Inception Labs, the company that released the model, co-authored a paper in late 2023 (revised in 2024), titled "Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution." The authors tested their proposed method on small autoregressive language modeling tasks, and claimed promising results:
Paper: https://arxiv.org/abs/2310.16834
Code: https://github.com/louaaron/Score-Entropy-Discrete-Diffusion
Blog post: https://aaronlou.com/blog/2024/discrete-diffusion/
It's possible -- in fact, likely -- that the new model is a much larger, more refined version of the small model presented and tested in that paper.