• sunbytes@lemmy.world
    link
    fedilink
    arrow-up
    13
    arrow-down
    4
    ·
    edit-2
    4 months ago

    I bet AI detection is going to get a lot better over time.

    I wonder if there’s going to be retrospective testing of theses as time goes on.

    Could really damage some careers down the line.

    Edit: guys, retrospective testing means it was done later (i.e. with a more up to date AI detector).

    • pflanzenregal@lemmy.world
      link
      fedilink
      arrow-up
      27
      ·
      4 months ago

      Once a detector is good, you can train a model to adjust its outputs to cause false negatives from the detector. Then the cycle repeats. It’s a cat and mouse game basically.

      The only proper way I see is a system that is based ob cryptographic signatures. This ia easier said than done ofc.

      • sunbytes@lemmy.world
        link
        fedilink
        arrow-up
        4
        ·
        4 months ago

        Yeah but if your wrote your thesis in 2024, and the detector is run on it in 2026…

        You’re probably busted.

        It’s not like you’ll re-write your thesis with every major ChatGPT release.

        • TehWorld@lemmy.world
          link
          fedilink
          arrow-up
          3
          arrow-down
          1
          ·
          4 months ago

          Are you expecting that the for-profit college will go back and retroactively rescind degrees? What’s the end-game for re-running the thesis?

          • Dinsmore@sh.itjust.works
            link
            fedilink
            arrow-up
            6
            ·
            4 months ago

            It likely won’t be done at scale, but let’s say you are wildly successful and are now in line for a high-value position, where vetting is common. Might look pretty bad if you fabricated your whole thesis. Recently, Bill Ackman basically bullied several schools into firing their head administrators on the pretense of not citing sources correctly in their thesis papers.

          • sunbytes@lemmy.world
            link
            fedilink
            arrow-up
            3
            ·
            4 months ago

            It could be a new level added to the peer review of work. Nothing to do with the university. Just “other professionals”.

            A thesis isn’t just an exam, it’s a real scientific paper.

            And usually claims is contents as fact, which can be referenced by others as fact.

            And absolutely should be open to scrutiny so long as it is relevant.

            • TehWorld@lemmy.world
              link
              fedilink
              arrow-up
              1
              ·
              4 months ago

              Great points. Note: I’m not arguing against it as a concept. I’m just skeptical that it’ll happen, and even if it did, there wouldn’t likely be terrible consequences for the accused, especially as that’s what science is… new facts change the outcome vs choosing an outcome and matching facts to it.

    • Turun@feddit.de
      link
      fedilink
      arrow-up
      6
      ·
      4 months ago

      I bet AI detection is going to get a lot better over time.

      I doubt it. ChatGPT 3.5 is good enough to rewrite small snippets of text with better phrasing, ChatGPT 4.0 can write a paragraph if given enough support. Good enough as in "the output is indistinguishable from what a human would have written.

      Of course you can do even more with the currently available tools - and get found out.

      There is a way to make AI generated text detectable: by slightly pushing the output towards a consistent pattern a detector can reliably judge long pieces of text as AI generated.
      Imagine if the AI is biased towards consecutive words starting with consecutive letters of the alphabet (e.g. “a blue car” instead of “a navy vehicle”.). Not strongly biased, but enough so that when there are 1000 words you can look at the probability of consecutive words starting with consecutive letters of the alphabet and get a clear result.

      There are two problems though: this only works with proprietary systems and only with long texts.

      • sunbytes@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        4 months ago

        If something was written by V3 and then published, that text doesn’t get updated every time a new version of chatGPT comes out.

        The text isn’t dynamic.

        • Turun@feddit.de
          link
          fedilink
          arrow-up
          1
          ·
          4 months ago

          Yes, but at some point it doesn’t matter. The AI is trained to replicate human writing. There will be a point where it becomes so good that the result is a perfect replica, where it is indistinguishable from human text. I.e. even a perfect detector will not be able to confidently declare it as AI written, not ever. Because there is no difference.

    • Alsjemenou@lemy.nl
      link
      fedilink
      Nederlands
      arrow-up
      4
      arrow-down
      1
      ·
      4 months ago

      Or we’re going the other way and just accept it as a tool for performing tasks that would otherwise take too much time.

      Granted that it makes the problem of teaching students the basics even more important.

    • tearsintherain@leminal.space
      link
      fedilink
      arrow-up
      1
      ·
      4 months ago

      But over time looks like the snake eating it’s own tail as AI iterates over everything. Someone will have to create fuzzy AI to dilute the writing down.