Frank Heijkamp on Nostr: This is a feature. Initially Large Language Models did not generate anything ...

This is a feature. Initially Large Language Models did not generate anything interesting, they got stuck fairly quickly. This was changed by 'relaxing' the parameters a little. Technically they are called weights and bias parameters. In layman terms, basically instructing the LLM to keep going, even if the following phrases were loosely related. Basically telling the LLM to keep pushing if it measures it is about to get stuck and 'allow' it to associate loosely in order to not get stuck.