Public research lab releases an open weights language model

A publicly funded research lab has released a language model with 70 billion parameters and published the weights under a permissive licence.

The lab describes the training run in a technical report that names the data sources it used.

According to the report, the run finished on a state owned cluster, and the lab intends to publish the training code as well.