• TheBlackLounge@lemmy.world
      link
      fedilink
      English
      arrow-up
      24
      ·
      edit-2
      11 months ago

      Distilling has been around since forever. It’s a legitimate technique that can give you a better model depending on your needs.

      OpenAI does it too to improve its models.

    • Zeth0s@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      11 months ago

      It’s like pupils learning from teachers, with the exception that there’s a single teacher in the whole world