Re: [paho-dev] AI contributions - policy needed?

[Greg Troxel <gdt@xxxxxxxxxx>]

"james.parrott via paho-dev" <paho-dev@xxxxxxxxxxx> writes:

> I'm not sure the problem is AI per se - I do use it myself. The
> problem's going to become more akin to spam, and 'contributers' that
> feel suspiciously like chatbots when you interact with them. I guess
> I'm not pushing for a ban, I'm just pushing for an understanding, that
> there's no longer an expectation, that any PR submitted without the
> tests being run on the fork, must be read.

I don't think any project has ever had a rule that all PRs must be
reviewed to any particular standard.  I think it's always been the case
that a reviewer can say quickly that the contribution doesn't meet
standards or isn't in condition such that expenditure of review
resources is reasonable.

It seems easy to have some text to paste in to anything that looks
suspicious

  Please explain if this PR contains any LLM-generated text.

  Please review the contributor guidelines and explain what testing you
  have done.

  Please close this PR and rebsubmit when ready, with text and code
  changes that are minimal and concise, in the way a human who
  understands the codebase would craft them.

or something like that.  The example of many PRs in a few hours that on
10 minute reading would seem to be properly many fewer and organized
better is the kind of thing that should just be rejected with a note
that the human should re-organize and trim.

There is a longstanding concept of "entry criteria" to inpsection
(inspection ~== formal/careful peer review) as a software engineering
process.

  https://archive.org/details/softwareinspecti0000gilb

The details are paged out, but the basic concept is that cursory,
high-speed quality checks (by a human, back then) must show that the
code/whatever is good enough that it's a sensible use of resources to
conduct an inpsection.

We haven't really dug into this in the open source world because
pre-LLMs, incoming PRs were often reasonable enough, or were few in
number.


It would also be reasonable to require that any PR that seems to use LLM
must have a CLA on file before any review at all.  Or perhaps that PRs
may only be submitted if CLA is on file, except for one PR that does not
involved LLM use.  Humans starting out often submit a PR before CLA, and
wheen it's thoughtful and just one, that's fine.