Home » Eclipse Projects » EGit / JGit » Patch handling process and mylyn?
| Patch handling process and mylyn? [message #3188] |
Mon, 20 April 2009 01:44  |
Eclipse User |
|
|
|
Originally posted by: alex_blewitt.yahoo.com
Having put a few patches together on Egit (and getting used to Git in
general, so please excuse any noob mistakes I make ...) I wanted to bring
up a difference between the way that patches are handled via the *git
review process and Eclipse.
Patches, as described in the SUBMITTING_PATCHES guide
(http://repo.or.cz/w/egit.git?a=blob;f=SUBMITTING_PATCHES;hb=HEAD)
involves emailing the patch to the current (2) maintainers, plus a Cc to
the (shared) *git mailing list. At some point, once they move past a
review, they presumably get folded upstream.
Eclipse, on the other hand, tends to use mail for discussions but patches
attached to bugs. Some projects even formalise it as part of the process
for comitters, not just contributions.
So I'm trying to find out what the plan is should things move to egit;
presumably, the developer discussions would move off the
git@vger.kernel.org mailing list and onto the eclipse.egit-dev mailing
list? And would patches start to flow via bug attachments instead?
Secondly, would we be able to take advantage of the nature of DVCS to
leverage applying patches in any way, like encouraging people to put
patches up via (say) github and then pulling in, or would that cause more
complications?
For those new to Eclipse/Mylyn, I've blogged about how to set up Google
Code from Mylyn at
http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and am
currently using this to track the issues I've raised/commented on. I'd be
interested if any of the existing or interested Egit participants find
this useful.
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #3222 is a reply to message #3188] |
Mon, 20 April 2009 06:45  |
|
Alex Blewitt schrieb:
> So I'm trying to find out what the plan is should things move to egit;
> presumably, the developer discussions would move off the
> git@vger.kernel.org mailing list and onto the eclipse.egit-dev mailing
> list? And would patches start to flow via bug attachments instead?
That's the way given by the Eclipse development process. Patches need to
be captured in Bugzilla bugs. This helps in tracking the IP log using
Bugzilla keywords and flags.
Of course, Bugzilla could CC the mailing list and discussions could
happen right away.
> Secondly, would we be able to take advantage of the nature of DVCS to
> leverage applying patches in any way, like encouraging people to put
> patches up via (say) github and then pulling in, or would that cause more
> complications?
I think pull is very problematic because of IP reasons. You never know
what you pull until you really pulled it. A patch is better because it
can be reviewed and clarified upfront.
-Gunnar
--
Gunnar Wagenknecht
gunnar@wagenknecht.org
http://wagenknecht.org/
|
|
| |
| Re: Patch handling process and mylyn? [message #3287 is a reply to message #3255] |
Mon, 20 April 2009 19:30 |
|
David Green schrieb:
> My understanding is that the terms and conditions of using Eclipse
> Bugzilla ensure that attachments are IP-clean.
No, using Bugzilla only ensures that the Eclipse.org website terms and
conditions apply. This ensures that all attachments are contributed
under EPL. However, the submitter/contributor still needs to confirm
that he wrote all the code and is allowed to contribute it. Sometimes an
employer may need to confirm as well. Even then, for some contributions
CQs may need to be filedas well for the legal team to review the
submitted code.
-Gunnar
--
Gunnar Wagenknecht
gunnar@wagenknecht.org
http://wagenknecht.org/
|
|
|
| Re: Patch handling process and mylyn? [message #3321 is a reply to message #3188] |
Mon, 20 April 2009 20:16 |
 Robin Rosenberg
Messages: 332
Registered: July 2009 |
Senior Member |
|
|
Alex Blewitt wrote:
> Having put a few patches together on Egit (and getting used to Git in
> general, so please excuse any noob mistakes I make ...) I wanted to bring
> up a difference between the way that patches are handled via the *git
> review process and Eclipse.
>
> Patches, as described in the SUBMITTING_PATCHES guide
> (http://repo.or.cz/w/egit.git?a=blob;f=SUBMITTING_PATCHES;hb=HEAD)
> involves emailing the patch to the current (2) maintainers, plus a Cc to
> the (shared) *git mailing list. At some point, once they move past a
> review, they presumably get folded upstream.
If either Shawn or I think the patches are good we apply, sign off them and
push to the official repository.
> Eclipse, on the other hand, tends to use mail for discussions but patches
> attached to bugs. Some projects even formalise it as part of the process
> for comitters, not just contributions.
Projects tend to use e they feel comfortable with. Sometimes
there is an explicit process. The Linux kernel has a process to ensure
IP cleanliness whereby author and maintainers sign off the patches.
> So I'm trying to find out what the plan is should things move to egit;
> presumably, the developer discussions would move off the
> git@vger.kernel.org mailing list and onto the eclipse.egit-dev mailing
> list? And would patches start to flow via bug attachments instead?
Good question. There is also the question of whether JGit can continue
to coexist under the same roof (with different licenses) or if we need
to split them and different homes, bug trackers. No decisions have been
made yet. In any case we'd move off the git mailing list.
Our current problem is that a mailing list is much better for discussing
patches than a bug tracker, but a bug tracker is much better for tracking
bugs that no one is actively working on. Hence we add a report to the
tracker for problems we find but probably won't fix right away, but
post fixes with or without bug reports, to the forum where lazy people can
read and comment without spending any time on mechanical processes like
copying and pasting.
Another problem we find as projects reach more contributors is that we find
them in twisted, environments with obstructed internet access, an
substandard e-mail (policy enforced) clients that mangle content.
> Secondly, would we be able to take advantage of the nature of DVCS to
> leverage applying patches in any way, like encouraging people to put
> patches up via (say) github and then pulling in, or would that cause more
> complications?
The reason we don't just pull and merge trees from contributors is that
we want to sign-off the patches to mark them as IP-clean. One could do
things differently. Mailing patches is very easy since it require only the
infrastructure that everybody has anyway.
Pulling complete branches is useful for getting quick access to code
for testing and for merging subsystems, i.e. branches where subsystem
maintainers have signed off each and every patch.
> For those new to Eclipse/Mylyn, I've blogged about how to set up Google
> Code from Mylyn at
> http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and am
> currently using this to track the issues I've raised/commented on. I'd be
> interested if any of the existing or interested Egit participants find
> this useful.
Is that setup different from what what is described in the Egit wiki?
-- robin
|
|
|
| Re: Patch handling process and mylyn? [message #3353 is a reply to message #3321] |
Tue, 21 April 2009 13:06 |
| Eclipse User |
|
|
|
Originally posted by: alex_blewitt.yahoo.com
>Robin Rosenberg wrote:
>> Alex Blewitt wrote:
>> Eclipse, on the other hand, tends to use mail for discussions but patches
>> attached to bugs. Some projects even formalise it as part of the process
>> for comitters, not just contributions.
>
> Projects tend to use e they feel comfortable with. Sometimes
> there is an explicit process. The Linux kernel has a process to ensure
> IP cleanliness whereby author and maintainers sign off the patches.
Right, but my point of starting this thread is that there is a current
disconnect between how the e/jgit projects operate and the way that
projects hosted at Eclipse operate. There's probably more stringent
processes relating to IP since the Eclipse Foundation host the code and so
need to guarantee a blood-line to descendants.
>> And would patches start to flow via bug attachments instead?
>
> Good question. There is also the question of whether JGit can continue
> to coexist under the same roof (with different licenses) or if we need
> to split them and different homes, bug trackers. No decisions have been
> made yet. In any case we'd move off the git mailing list.
I'm sure part of the review/proposal process will cover that. Maybe it's
possible for two different licenses to co-exist under the same project;
certainly, one of the sign-off conditions is that each plug-in has its own
LICENSE file. So it's quite likely that the JGit would have a BSD-esque
license (EDL?) whilst the EGit would have EPL. Whether that would mean to
projects (eclipse.jgit) or whether they'd be two modules under the same
project (like http://www.eclipse.org/dsdp/tm/ which contains both) is
something for the Eclipse Foundation to comment on.
> Our current problem is that a mailing list is much better for discussing
> patches than a bug tracker
Is it much better because that's what you're used to, or are there real
use cases that you have in mind? After all, bug tracking can work for a
back-and-forth of discussion and patches, as seen (e.g.) here:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=272124
> But a bug tracker is much better for tracking
> bugs that no one is actively working on. Hence we add a report to the
> tracker for problems we find but probably won't fix right away, but
> post fixes with or without bug reports
This has a fairly significant downside, in that it forces everyone to be a
member of the mailing list in order to contribute or discuss. I've just
removed myself from the Git mailing list because there's way too much
noise to be useful to me personally - and with everyone mailing patches,
my inbox was filling up significantly faster just through membership of
this list. In addition, almost all of the mail was (to me) junk; the only
items I was interested in was feedback about the patches I was sending in.
And yet the SUBMITTING_PATCHES guide requires sending the patch to the git
mailing list to co-operate.
That's a significantly higher burden than any other Eclipse project; quite
aside from the Eclipse process that Gunnar alluded to (which requires
patches to go through bugzilla), it means that contributors must be
expected to bear the full brunt of the spam that comes in. For some
projects, especially perhaps where there's a hard-core set of developers,
it might be modus operandi, but it precludes anyone else from contribution
occasionally.
> to the forum where lazy people can read and comment without spending
> any time on mechanical processes like copying and pasting.
I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=272970 to suggest
that this capability could be built into mylyn, which it isn't at present.
But note that having the attachment can then provide other functionality
such as a web-based diff:
https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=diff
(Incidentally, this should probably be something supported in the back end
by the Git provider in order to encourage adoption of Git by other
projects)
> Another problem we find as projects reach more contributors is that we find
> them in twisted, environments with obstructed internet access, an
> substandard e-mail (policy enforced) clients that mangle content.
Indeed. I don't think I've yet managed to successfully send mail other
than by cut, paste and pipe to sendmail. And even then the spam filter at
vger bounced it back.
> Mailing patches is very easy since it require only the
> infrastructure that everybody has anyway.
True, but then everyone has web access too. And there are client
complicating factors, which you don't get with the web. Plus, some
organisations have policies specifically for mailing code (I know of some
organisations whose outgoing mail filters check for code originating from
inside and explicitly prohibit the mail in case there's internal code
leaks). Not to mention archiving of e-mails, which may be required for
regulatory purposes.
>> For those new to Eclipse/Mylyn, I've blogged about how to set up Google
>> Code from Mylyn at
>> http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and am
>> currently using this to track the issues I've raised/commented on. I'd be
>> interested if any of the existing or interested Egit participants find
>> this useful.
> Is that setup different from what what is described in the Egit wiki?
Which setup are you referring to? I could find no mention of mylyn on the
Egit wiki. Perhaps you could provide a URL to the page you mean?
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #3387 is a reply to message #3353] |
Tue, 21 April 2009 19:37 |
Shawn O. Pearce
Messages: 82
Registered: July 2009 |
Member |
|
|
Alex Blewitt wrote:
>> Robin Rosenberg wrote:
>
> Right, but my point of starting this thread is that there is a current
> disconnect between how the e/jgit projects operate and the way that
> projects hosted at Eclipse operate.
Yes. That disconnect has more to do with our heritage than just wanting
to be different for the sake of being different.
EGit started out in early 2006 as a project away from Eclipse, because
to be honest, I didn't know where to begin to propose a project for
hosting at Eclipse.org. And now that we are working on the process, its
simply way too much effort. I never would have proposed EGit to
Eclipse.org back in 2006. I advertised EGit on git@vger because other
Git users were there, and other frontends (QGit, gitk) were advertised
there every once in a while. We stuck to git@vger because people
sometimes mention the plugin, or post patches there.
>>> And would patches start to flow via bug attachments instead?
>> Our current problem is that a mailing list is much better for discussing
>> patches than a bug tracker
>
> Is it much better because that's what you're used to, or are there real
> use cases that you have in mind?
From what I have seen, doing code reviews of proposed patches falls
into three levels of convenience for the reviewer:
* Most Convenient: Gerrit Code Review; Reitveld; Google Mondrian
Patches can be commented inline directly in code. Drafts can be
written, and later emailed once polished.
Its easy to bounce around between files, or commits in the same series.
Its easy to see interdiffs between different versions of the same
proposed change.
Its easy to see your prior comments inline on the left pane, while
looking at how the patch author addressed them in the right pane.
Its easy to download and patch locally, to compile, execute, or view
with other tools.
For a contributor, its easy to upload a proposed change. Gerrit
supports git push directly over SSH. Reitveld has a command line Python
script. Google Mondrian has its own upload script (not that Mondrian
matters to anyone but Googlers, but its dead simple to use).
* Somewhat Convenient: Email
Its easy to reply. Your MUA quotes the code, so you can insert messages
inline.
Its easy to bounce around, or save drafts, just use your MUAs navigation
features. Sadly, I'm not as good with mutt as I should be for that to
really work well for me; I find Gerrit to be much easier.
Its easy to download and patch locally, given git-am.
Where I find email lacking is its difficult to compare the current
latest version of a change against my prior review comments, to remind
myself of what I said, and if the author addressed my concerns.
For a contributor, its easy to use git send-email, if you can tame that
beast. Once you get it configured right for your MTA, its easy to fire
off a 25 patch series to the maintainers.
* Least Convenient: Bug Tracker
Yea, sure, attach a patch file to a bug tracker. You can use comments
on the bug to have a conversation.
But there's no automated quoting. There's no easy way to draw attention
to a particular line code.
There's no easy way to interdiff the changes.
There's no easy way to download a long series (5+ commits). If each is
its own bug, you need to download one patch file per bug, then apply
them in order. Its easier in a MUA to save a chunk of emails to a
single mbox and shove it through git-am. Or in Gerrit to just issue
"git pull review refs/changes/nn/nnnn/1" to merge the entire thing.
For a contributor, its easy to attach one patch file, but at 2 patches,
I draw the line. I abhor patching Apache MINA SSHD because I need to
generate patch files and attach them to discrete bugs in their JIRA bug
tracking system. Its no less then 7 mouse clicks PER PATCH. I'll never
contribute more than a minor bug fix to MINA SSHD as a result. The
effort required to upload the F*@!!@* change is overshadowed by the
value the patch might bring to me.
So one thing I'm _really_ not looking forward to with the move to
Eclipse.org is the IP requirement that all changes not committed
directly to the repository go through Bugzilla. I think its going to be
a higher burden for contributors. I think its going to cause pain for
Robin and I with regards to reviewing code prior to it being committed.
I really like the fact that I can't just commit whatever I damn well
please to egit.git. I like having Robin sanity check my work before it
enters the repository. I'm not perfect, I know Robin has saved me from
myself on more than one occasion.
> This has a fairly significant downside, in that it forces everyone to be
> a member of the mailing list in order to contribute or discuss.
Not true. git@vger is open-posting. Anyone can post, without being
subscribed. The list has a Reply-To-All policy, so that people who are
not subscribed, but who are listed on the To/CC fields still receive the
thread.
> That's a significantly higher burden than any other Eclipse project;
Obtaining an Eclipse.org Bugzilla account, with a new unique
username/password pair, is a lot more effort than drive-by-patching to
git@vger.
> I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=272970 to suggest
> that this capability could be built into mylyn, which it isn't at
> present. But note that having the attachment can then provide other
> functionality such as a web-based diff:
>
> https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=diff
>
> (Incidentally, this should probably be something supported in the back
> end by the Git provider in order to encourage adoption of Git by other
> projects)
Yes. Look at Gerrit Code Review[1]. Its a JGit backed, web based code
review system. It uses Apache MINA SSHD to provide git fetch service
for clients to easily download proposed changes, and git push service to
upload new changes for review. Currently it depends upon C git to
produce diffs, but as soon as JGit has Good Enough(tm) diff support,
Gerrit will move to use it.
Gerrit may actually be favorable to Eclipse.org. Its entire APLv2 or
BSD (JSch and JGit). It can be configured to enforce contributors to
sign agreements prior to being able to post code. Those agreements can
be electronic click-through, or paper based forms that require a clerk
to ACK in the system upon receipt. As a result, as a user, you know in
advance that everything on the review server has been promised to be
IP-clean by the author, and that they have completed a sufficient
contribution agreement. Pulling code from there is therefore (in
theory) as safe as pulling code from the main repository.
[1] http://code.google.com/gerrit/
>> Another problem we find as projects reach more contributors is that we
>> find them in twisted, environments with obstructed internet access, an
>> substandard e-mail (policy enforced) clients that mangle content.
Yes. This is where Gerrit's requirement for SSH is actually a problem.
Its damn convenient to do "git push review HEAD:refs/for/master" in
order to propose a change, but if you are behind a firewall that blocks
outgoing encrypted SSH connections on port 29418, you are SOL.
And thus email is very suitable.
But, usually such braindamaged firewalls have unrestricted HTTP and
HTTPS on port 80 and 443 and will proxy to almost any site out there.
Half of them don't even try to do MITM attacks on the SSL traffic and
thus bypass any content filtering that they otherwise do on port 80.
HTTP is the new TCP. *sigh*
|
|
|
| Re: Patch handling process and mylyn? [message #3420 is a reply to message #3353] |
Tue, 21 April 2009 19:58 |
Robin Rosenberg
Messages: 332
Registered: July 2009 |
Senior Member |
|
|
Alex Blewitt wrote:
>>Robin Rosenberg wrote:
>>> Alex Blewitt wrote:
>>> Eclipse, on the other hand, tends to use mail for discussions but
>>> patches attached to bugs. Some projects even formalise it as part of the
>>> process for comitters, not just contributions.
>>
>> Projects tend to use e they feel comfortable with. Sometimes
>> there is an explicit process. The Linux kernel has a process to ensure
>> IP cleanliness whereby author and maintainers sign off the patches.
>
> Right, but my point of starting this thread is that there is a current
> disconnect between how the e/jgit projects operate and the way that
> projects hosted at Eclipse operate. There's probably more stringent
> processes relating to IP since the Eclipse Foundation host the code and so
> need to guarantee a blood-line to descendants.
>
>>> And would patches start to flow via bug attachments instead?
>>
>> Good question. There is also the question of whether JGit can continue
>> to coexist under the same roof (with different licenses) or if we need
>> to split them and different homes, bug trackers. No decisions have been
>> made yet. In any case we'd move off the git mailing list.
>
> I'm sure part of the review/proposal process will cover that. Maybe it's
> possible for two different licenses to co-exist under the same project;
> certainly, one of the sign-off conditions is that each plug-in has its own
> LICENSE file. So it's quite likely that the JGit would have a BSD-esque
> license (EDL?) whilst the EGit would have EPL. Whether that would mean to
> projects (eclipse.jgit) or whether they'd be two modules under the same
> project (like http://www.eclipse.org/dsdp/tm/ which contains both) is
> something for the Eclipse Foundation to comment on.
>
>> Our current problem is that a mailing list is much better for discussing
>> patches than a bug tracker
>
> Is it much better because that's what you're used to, or are there real
> use cases that you have in mind? After all, bug tracking can work for a
> back-and-forth of discussion and patches, as seen (e.g.) here:
>
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=272124
>
>> But a bug tracker is much better for tracking
>> bugs that no one is actively working on. Hence we add a report to the
>> tracker for problems we find but probably won't fix right away, but
>> post fixes with or without bug reports
>
> This has a fairly significant downside, in that it forces everyone to be a
> member of the mailing list in order to contribute or discuss. I've just
> removed myself from the Git mailing list because there's way too much
> noise to be useful to me personally - and with everyone mailing patches,
> my inbox was filling up significantly faster just through membership of
> this list. In addition, almost all of the mail was (to me) junk; the only
> items I was interested in was feedback about the patches I was sending in.
> And yet the SUBMITTING_PATCHES guide requires sending the patch to the git
> mailing list to co-operate.
It's correct that the Git ML is too high volume for widespread use specific
to EGit/JGit, but we started out that way with very few complaints. You
can submit patches to the list without being subscribed and by convention
people reply directly to all participants in a thread, so if you post
something you get all feedback.
> That's a significantly higher burden than any other Eclipse project; quite
> aside from the Eclipse process that Gunnar alluded to (which requires
> patches to go through bugzilla), it means that contributors must be
> expected to bear the full brunt of the spam that comes in. For some
> projects, especially perhaps where there's a hard-core set of developers,
> it might be modus operandi, but it precludes anyone else from contribution
> occasionally.
>
>> to the forum where lazy people can read and comment without spending
>> any time on mechanical processes like copying and pasting.
>
> I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=272970 to suggest
> that this capability could be built into mylyn, which it isn't at present.
> But note that having the attachment can then provide other functionality
> such as a web-based diff:
>
> https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=diff
>
> (Incidentally, this should probably be something supported in the back end
> by the Git provider in order to encourage adoption of Git by other
> projects)
>
>> Another problem we find as projects reach more contributors is that we
>> find them in twisted, environments with obstructed internet access, an
>> substandard e-mail (policy enforced) clients that mangle content.
>
> Indeed. I don't think I've yet managed to successfully send mail other
> than by cut, paste and pipe to sendmail. And even then the spam filter at
> vger bounced it back.
Hmmm.
>> Mailing patches is very easy since it require only the
>> infrastructure that everybody has anyway.
>
> True, but then everyone has web access too. And there are client
> complicating factors, which you don't get with the web. Plus, some
> organisations have policies specifically for mailing code (I know of some
> organisations whose outgoing mail filters check for code originating from
> inside and explicitly prohibit the mail in case there's internal code
> leaks). Not to mention archiving of e-mails, which may be required for
> regulatory purposes.
It's not the ultimate, it's been good because it didn't require any
infrastructure setup.
BTW, Now I saw something I didn't see before. The Eclipse
bugzilla has an Edit Attachment as comment, which looks like the exact
same thing. I haven't tried pressing Submit, though. If it works as
I think, I'll shut up...
Github also has line-level commenting, which looks nice and then there's
Gerrit2.
>>> For those new to Eclipse/Mylyn, I've blogged about how to set up Google
>>> Code from Mylyn at
>>> http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and
>>> am currently using this to track the issues I've raised/commented on.
>>> I'd be interested if any of the existing or interested Egit participants
>>> find this useful.
>> Is that setup different from what what is described in the Egit wiki?
>
> Which setup are you referring to? I could find no mention of mylyn on the
> Egit wiki. Perhaps you could provide a URL to the page you mean?
"Featured Links" Configuring Mylin. Leading to
http://code.google.com/p/egit/wiki/ConfiguringMylyn. The generice web
connector and hasn't been very useful to me since it hasn't been full
roundtrip. I think bugzilla will be more useful to us.
-- robin
|
|
|
| Re: Patch handling process and mylyn? [message #3454 is a reply to message #3420] |
Wed, 22 April 2009 11:49 |
| Eclipse User |
|
|
|
Originally posted by: alex_blewitt.nospam.yahoo.com
Robin Rosenberg wrote:
> It's correct that the Git ML is too high volume for widespread use specific
> to EGit/JGit, but we started out that way with very few complaints. You
> can submit patches to the list without being subscribed and by convention
> people reply directly to all participants in a thread, so if you post
> something you get all feedback.
Good to know.
>>> Is that setup different from what what is described in the Egit wiki?
>>
>> Which setup are you referring to? I could find no mention of mylyn on the
>> Egit wiki. Perhaps you could provide a URL to the page you mean?
>
> "Featured Links" Configuring Mylin. Leading to
> http://code.google.com/p/egit/wiki/ConfiguringMylyn. The generice web
> connector and hasn't been very useful to me since it hasn't been full
> roundtrip. I think bugzilla will be more useful to us.
Right, I was looking in the wrong wiki. The 'wiki' link on the main page
(http://code.google.com/p/egit/) takes you to a git.or.cz site
(http://git.or.cz/gitwiki/EclipsePlugin) which doesn't have any search
results returned for Mylyn, which is why I asked.
There are a couple of differences, yes:
1) It uses 'can=2' instead of 'can=1' which only downloads the open issues
2) It adds 'can' and 'search' parameters which you can change for each
individual query, instead of just having one. I wrote about the kinds of
search fields that you can use e.g. owner:shawn.pearce or
owner:jonas.fonseca priority:low.
You can change the (code.google.com) wiki to read:
Query request URL:
${serverUrl}/csv?can=${can}&colspec=ID+Status+Type+Owner +Summary&q=${search}
Query pattern:
^"({Id}[0-9]+?)","({Status}.*?)","({Type}.*?)","({Owner}.*?) ","({Description}.
*?)"$
Parameter:
can (value 2)
Parameter:
search (no value)
There's more explaination on my post about it.
http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #3488 is a reply to message #3387] |
Wed, 22 April 2009 11:49 |
| Eclipse User |
|
|
|
Originally posted by: alex_blewitt.nospam.yahoo.com
Shawn Pearce wrote:
> Alex Blewitt wrote:
>>> Robin Rosenberg wrote:
>>
>> Right, but my point of starting this thread is that there is a current
>> disconnect between how the e/jgit projects operate and the way that
>> projects hosted at Eclipse operate.
>
> Yes. That disconnect has more to do with our heritage than just wanting
> to be different for the sake of being different.
OK, that's good to know.
> * Somewhat Convenient: Email
>
> Its easy to reply. Your MUA quotes the code, so you can insert messages
> inline.
>
> Its easy to bounce around, or save drafts, just use your MUAs navigation
> features. Sadly, I'm not as good with mutt as I should be for that to
> really work well for me; I find Gerrit to be much easier.
>
> Its easy to download and patch locally, given git-am.
A goal of EGit should be to make it as easy to apply patches via Mylyn
as it currently is with SVN and CVS. That's the modus operandi for Eclipse
projects, and would be a big win for adoption.
> Where I find email lacking is its difficult to compare the current
> latest version of a change against my prior review comments, to remind
> myself of what I said, and if the author addressed my concerns.
One of the benefits of an issue tracker is that you log all the comments with
the patch, and it's persisted. So unlike losing mail from months ago, you've
got all the history with the same item.
> * Least Convenient: Bug Tracker
>
> Yea, sure, attach a patch file to a bug tracker. You can use comments
> on the bug to have a conversation.
>
> But there's no automated quoting. There's no easy way to draw attention
> to a particular line code.
Perhaps you've not seen the 'quote attachment' feature of bugzilla?
If you view the attachment, you get an 'Edit attachment as comment' button:
https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=edit
That will quote the entire attachment in exactly the same way as a 'reply'
would in your MUA.
> There's no easy way to download a long series (5+ commits). If each is
> its own bug, you need to download one patch file per bug, then apply
> them in order.
I think we can be intelligent here to try and get a good working practice.
Whilst it's true that there's generally one patch per bug in Eclipse, tools
like Mylyn can make applying those patches easier.
One big learning curve is going to be the small nature of patches, or rather,
the combination of patches that are generated when you're off from master.
Perhaps there are other ways of approaching this, like attaching a
zip-of-patches as an attachment to bugzilla? If a change is logically for one
item, then does it make sense to attach them to different bugs?
> For a contributor, its easy to attach one patch file, but at 2 patches,
> I draw the line. I abhor patching Apache MINA SSHD because I need to
> generate patch files and attach them to discrete bugs in their JIRA bug
> tracking system.
That sounds like an issue with the process, rather than the use of an issue
tracking system.
How can we make it easier? One idea might be to have a multi-patch file, which
contains all the patches but in a single file (compressed or concatenated).
After all, one of your cited examples is that 'git-am' makes it easy. What do
we need to do to 'Eclipse/Mylyn' to make it easy? That seems to be the key
question.
> So one thing I'm _really_ not looking forward to with the move to
> Eclipse.org is the IP requirement that all changes not committed
> directly to the repository go through Bugzilla. I think its going to be
> a higher burden for contributors. I think its going to cause pain for
> Robin and I with regards to reviewing code prior to it being committed.
I wouldn't have put it at a higher burden for contributors over any other
Eclipse project.
> I really like the fact that I can't just commit whatever I damn well
> please to egit.git. I like having Robin sanity check my work before it
> enters the repository. I'm not perfect, I know Robin has saved me from
> myself on more than one occasion.
I'll all for code reviews. But code reviews don't have to be based on mail -
in fact, the bugzilla has a 'reviewer' which can be set to flag someone as
having reviewed the patch.
> Yes. Look at Gerrit Code Review[1]. Its a JGit backed, web based code
> review system. It uses Apache MINA SSHD to provide git fetch service
> for clients to easily download proposed changes, and git push service to
> upload new changes for review. Currently it depends upon C git to
> produce diffs, but as soon as JGit has Good Enough(tm) diff support,
> Gerrit will move to use it.
> [1] http://code.google.com/gerrit/
Did you mean http://code.google.com/p/gerrit/ ?
I can see something like that being useful when there are many Git projects.
It might be difficult in getting support for running an additional tool by the
Eclipse webmasters though. I think there'd probably need to be a high
penetration of Git projects to make it worthwhile.
> Yes. This is where Gerrit's requirement for SSH is actually a problem.
> Its damn convenient to do "git push review HEAD:refs/for/master" in
> order to propose a change, but if you are behind a firewall that blocks
> outgoing encrypted SSH connections on port 29418, you are SOL.
Right. In fact, speaking as someone who is behind a firewall most of the time,
these kind of requirements are more of a burden than you might imagine. Bear
in mind that a lot of Eclipse support comes from member companies who do have
firewalls and only allow HTTP/S out.
> But, usually such braindamaged firewalls have unrestricted HTTP and
> HTTPS on port 80 and 443 and will proxy to almost any site out there.
> Half of them don't even try to do MITM attacks on the SSL traffic and
> thus bypass any content filtering that they otherwise do on port 80.
>
> HTTP is the new TCP. *sigh*
True. A lof of the perceived advantages of SVN was that you could access it
over HTTP and thus be able to use it from such sites.
http://alblue.blogspot.com/2007/11/eclipse-svn-over-https.ht ml
We should strive to make all processes work at a minimum over HTTP/S in order
to get the biggest win. Any procses that involves SSH over whatever port isn't
likely to be something that gets a great deal of support IMHO.
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #3521 is a reply to message #3488] |
Wed, 22 April 2009 15:24 |
Shawn O. Pearce
Messages: 82
Registered: July 2009 |
Member |
|
|
Alex Blewitt wrote:
> A goal of EGit should be to make it as easy to apply patches via Mylyn
> as it currently is with SVN and CVS. That's the modus operandi for Eclipse
> projects, and would be a big win for adoption.
Ok, I'll have to look at how Mylyn does this with SVN and CVS then.
Because I haven't looked at it before. Hell, I haven't used SVN except
by git-svn in years. And I haven't used CVS at all in more than 6 or 7
years.
Long-term, I should consider connecting Gerrit Code Review with Mylyn.
A number of folks who use Gerrit also use Eclipse.
>> But there's no automated quoting. There's no easy way to draw attention
>> to a particular line code.
>
> Perhaps you've not seen the 'quote attachment' feature of bugzilla?
No, I hadn't seen that before.
But editing text in a browser text widget. *sigh* Now I need to go
find a "vi" textarea replacement plugin for Firefox. Someone out there
must have written one.
I should have also mentioned Review Board yesterday in the "really easy"
category. I haven't used it myself, but people like it.
> One big learning curve is going to be the small nature of patches, or rather,
> the combination of patches that are generated when you're off from master.
> Perhaps there are other ways of approaching this, like attaching a
> zip-of-patches as an attachment to bugzilla? If a change is logically for one
> item, then does it make sense to attach them to different bugs?
Hmmph. One one hand you raise the 'quote attachment' feature of
bugzilla, and then on the other you suggest making the attachment a ZIP
archive? How would bugzilla quote that attachment for commenting?
>> For a contributor, its easy to attach one patch file, but at 2 patches,
>> I draw the line. I abhor patching Apache MINA SSHD because I need to
>> generate patch files and attach them to discrete bugs in their JIRA bug
>> tracking system.
>
> That sounds like an issue with the process, rather than the use of an issue
> tracking system.
Is the current Eclipse process really any better with Bugzilla?
Assuming the awesome CVS team provider and Mylyn are being used? For
non-committers?
>> So one thing I'm _really_ not looking forward to with the move to
>> Eclipse.org is the IP requirement that all changes not committed
>> directly to the repository go through Bugzilla. I think its going to be
>> a higher burden for contributors. I think its going to cause pain for
>> Robin and I with regards to reviewing code prior to it being committed.
>
> I wouldn't have put it at a higher burden for contributors over any other
> Eclipse project.
No, EGit's burden is the same for any other Eclipse project.
My point is, at present, it appears that the burden of *being* an
Eclipse project is higher than what we have enjoyed up until now from
the Git mailing list, and the Git tooling.
>> Yes. Look at Gerrit Code Review[1].
>
>> [1] http://code.google.com/gerrit/
>
> Did you mean http://code.google.com/p/gerrit/ ?
*sigh*. Yes.
> I can see something like that being useful when there are many Git projects.
> It might be difficult in getting support for running an additional tool by the
> Eclipse webmasters though. I think there'd probably need to be a high
> penetration of Git projects to make it worthwhile.
As you said, the penetration of Git projects needs to be enough to make
it worthwhile. The foundation webmasters have asked that Git replace an
existing VCS. If we replaced either CVS or SVN, I think the penetration
of projects is sufficient to justify Git on the server.
One of the requirements for that replacement to be successful is a good
team provider, aka EGit, that can be shipped with the platform, like the
CVS team provider is.
If Eclipse.org supports a Git repository, its going to need to support a
secure method of writing to that repository, one that can be reasonably
audited, like the current CVS over SSH.
At that point, you are talking about running Gitosis or a home-brewed
solution over SSH, like GitHub runs. Gerrit Code Review also provides
similar access control support. It can be used as a heavyweight
replacement for Gitosis, with the bonus of having an integrated code
review system.
So, the point I was trying to make was, if Eclipse.org does wind up
supporting Git, it will need to also explore Git server tools, at least
one of which includes what I consider to be a better code review
interface than Bugzilla's "quote attachment" feature. But, I am also
very biased.
>> Yes. This is where Gerrit's requirement for SSH is actually a problem.
>
> Right. In fact, speaking as someone who is behind a firewall most of the time,
> these kind of requirements are more of a burden than you might imagine. Bear
> in mind that a lot of Eclipse support comes from member companies who do have
> firewalls and only allow HTTP/S out.
Every time I hear that, I question the competency of the IT staff, or
the dedication of management to the contribution.
>> HTTP is the new TCP. *sigh*
>
> True. A lof of the perceived advantages of SVN was that you could access it
> over HTTP and thus be able to use it from such sites.
See. These sorts of companies block the CVS pserver port, because they
don't want code going in or out of the company. So SVN goes off and
embeds its protocol into HTTP, so that it can bypass any company's
firewall rules.
So now the IT staff is saying to management "we block those nasty code
sharing programs!" and meanwhile the engineers are running SVN over
HTTPS, defeating management's request that code not be shared.
If management actually blessed the staff consuming code and contributing
code back to eclipse.org, then IT should be able to write the firewall
rule to permit the access.
If IT can't, then either its because the management really isn't that
committed, or the IT staff can't write reasonable firewall rules that
balance the security of the internal network against the business needs
of the organization.
I'm sorry, but I find it hard to believe that a member company of
Eclipse, who is paying $20k-$250k/year just in membership dues, is so
uncommitted to Eclipse that they want to encourage their staff to punch
through their own firewalls by tunneling everything over HTTP.
> We should strive to make all processes work at a minimum over HTTP/S in order
> to get the biggest win. Any procses that involves SSH over whatever port isn't
> likely to be something that gets a great deal of support IMHO.
Unless Git suddenly grows support for a more efficient HTTP protocol for
both fetch and push, that's insane. And even then, I think its insane.
Embedding a data transfer protocol into HTTPS just to bypass
conservative firewalls is simply nuts.
|
|
|
| Re: Patch handling process and mylyn? [message #3554 is a reply to message #3521] |
Wed, 22 April 2009 23:48 |
| Eclipse User |
|
|
|
Originally posted by: alex_blewitt.nospam.yahoo.com
Shawn Pearce wrote:
> Alex Blewitt wrote:
>> One big learning curve is going to be the small nature of patches, or
rather,
>> the combination of patches that are generated when you're off from master.
>> Perhaps there are other ways of approaching this, like attaching a
>> zip-of-patches as an attachment to bugzilla? If a change is logically for
one
>> item, then does it make sense to attach them to different bugs?
>
> Hmmph. One one hand you raise the 'quote attachment' feature of
> bugzilla, and then on the other you suggest making the attachment a ZIP
> archive? How would bugzilla quote that attachment for commenting?
I'm trying to think aloud about how it would work. Having multiple patches
generated from a (say) git format-patch seems to be the tricky thing; so how
do you attach them to a bug? As multiple attachments? Or can we concatenate
them together into a multi-patch file somehow? It was more a way of
investigating what would be the best way to replicate the ease of applying
many patches without having to have many attachments/bugs (which as noted
earlier, above 2 makes it detrimental)
> Is the current Eclipse process really any better with Bugzilla?
> Assuming the awesome CVS team provider and Mylyn are being used? For
> non-committers?
It's actually pretty easy, yes. I've submitted a number of patches to a number
of different projects by finding, fixing, then submitting the patch to the bug
and leaving it to go from there.
> My point is, at present, it appears that the burden of *being* an
> Eclipse project is higher than what we have enjoyed up until now from
> the Git mailing list, and the Git tooling.
Possibly - but the goal of EGit is to make it much easier to use. Perhaps
instead of the reliance on external (mail-based) tooling, we should focus on
tight integration with Mylyn. Actually, I suspect most of the hard work is
already there, since Mylyn is the one that talks to back ends already.
http://wiki.eclipse.org/Mylyn_FAQ#Team_Support
The key here is that it's the tooling that needs enhancement, rather than the
back end particularly. As a case in point, when creating a CVS diff in Mylyn,
it's possible to dump that patch to the clipboard and then have Mylyn upload
that patch from the clipboard without ever hitting the disk.
>>> Yes. This is where Gerrit's requirement for SSH is actually a problem.
>>
>> Right. In fact, speaking as someone who is behind a firewall most of the
time,
>> these kind of requirements are more of a burden than you might imagine.
Bear
>> in mind that a lot of Eclipse support comes from member companies who do
have
>> firewalls and only allow HTTP/S out.
>
> Every time I hear that, I question the competency of the IT staff, or
> the dedication of management to the contribution.
Every time I hear that, I wonder to what extent the involvement with large
companies is :-)
> See. These sorts of companies block the CVS pserver port, because they
> don't want code going in or out of the company. So SVN goes off and
> embeds its protocol into HTTP, so that it can bypass any company's
> firewall rules.
That's a pretty naive assumption of the world. It's not just a case of
particular ports being blocked (or opened) - more often than not, the internal
network is completely unreachable by any protocol/port to the outside world
unless it's in the blessed zone (which itself is firewalled in and out, too).
I haven't yet worked for a company in the last 10 years whose machines
internally had an IP connection to the outside world; they were all on private
(10.x.x.x) subnets.
Access to the outside world is via e.g. mail (and Outlook is not great if you
want to preserve niceities like spacing, frankly) and HTTP proxies, which
forward requests on your behalf. And even then, firewalls don't just let
anything go through on 80 or 443; in fact, connecting via a locked-down HTTPS
proxy will only let you CONNECT to other machines on 443 in some cases. A few
places have, for those who need it, SOCKS access to the outside world, which
is more of an IP proxy.
> So now the IT staff is saying to management "we block those nasty code
> sharing programs!" and meanwhile the engineers are running SVN over
> HTTPS, defeating management's request that code not be shared.
Shared != consumed != contributions, though. I've found several bugs in my
time from Eclipse whilst at work, but I can't investigate and find the bug
there and then because I can't get to HEAD. Often, it involves side-stepping
the process to download a TGZ from one of the nightly builds or other such
time and bandwidth wasters (for all sides).
> If management actually blessed the staff consuming code and contributing
> code back to eclipse.org, then IT should be able to write the firewall
> rule to permit the access.
Again, a private IP address isn't routable from the outside internet. It's not
just a firewall thing.
> If IT can't, then either its because the management really isn't that
> committed, or the IT staff can't write reasonable firewall rules that
> balance the security of the internal network against the business needs
> of the organization.
The business needs of the organisation are way more set up to prevent e.g.
viruses and access to machines from the outside than anything else. And as
noted, it's not just a firewall thing - these machines are unroutable from the
outside world and can only get there through proxies.
> I'm sorry, but I find it hard to believe that a member company of
> Eclipse, who is paying $20k-$250k/year just in membership dues, is so
> uncommitted to Eclipse that they want to encourage their staff to punch
> through their own firewalls by tunneling everything over HTTP.
Still unroutable machines from the outside world.
> Unless Git suddenly grows support for a more efficient HTTP protocol for
> both fetch and push, that's insane. And even then, I think its insane.
> Embedding a data transfer protocol into HTTPS just to bypass
> conservative firewalls is simply nuts.
OK, so that we're clear one last time - it's not about port blocking. It's
about having an unroutable machine inside a corporate network, communicating
via a proxy, in order to speak to the outside world. It's not a case of 'can't
you open up a port' or incompetence in the case of network engineers.
http://en.wikipedia.org/wiki/Private_network
"They are also commonly used in corporate networks, which for security
reasons, are not connected directly to the internet, meaning globally routable
addresses are unnecessary. Often a proxy, SOCKS gateway, or similar is used to
provide restricted internet access to internal users. In both cases, private
addresses are seen as adding security to the internal network, since it's
impossible for an Internet host to connect directly to an internal system."
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #3587 is a reply to message #3554] |
Thu, 23 April 2009 00:58 |
Shawn O. Pearce
Messages: 82
Registered: July 2009 |
Member |
|
|
Alex Blewitt wrote:
> I'm trying to think aloud about how it would work. Having multiple patches
> generated from a (say) git format-patch seems to be the tricky thing; so how
> do you attach them to a bug? As multiple attachments? Or can we concatenate
> them together into a multi-patch file somehow?
Yes.
git format-patch --stdout ... >foo.mbox
git am foo.mbox
> Possibly - but the goal of EGit is to make it much easier to use. Perhaps
> instead of the reliance on external (mail-based) tooling, we should focus on
> tight integration with Mylyn. Actually, I suspect most of the hard work is
> already there, since Mylyn is the one that talks to back ends already.
Yes. Mik Kersten approach me about Mylyn during EclipseCon and wants to
help out with the integration effort. Connecting with the other tools
in the Eclipse environment is a great approach to making things
smoother. :-)
>> Every time I hear that, I question the competency of the IT staff, or
>> the dedication of management to the contribution.
>
> Every time I hear that, I wonder to what extent the involvement with large
> companies is :-)
:-)
How quickly I forget that I spent the last 5+ years at a Fortune 10
company. Their network was isolated. We used Eclipse. But yea, you're
right, we only had a brain-damaged HTTP proxy to the outside world.
I've been here at current job too long. Its a really big company.
Household name in pretty much any country. But we also really like the
Internet and I have pretty much unrestricted access from my desktop.
I'm spoiled. :-)
> It's not just a case of
> particular ports being blocked (or opened) - more often than not, the internal
> network is completely unreachable by any protocol/port to the outside world
> unless it's in the blessed zone (which itself is firewalled in and out, too).
> I haven't yet worked for a company in the last 10 years whose machines
> internally had an IP connection to the outside world; they were all on private
> (10.x.x.x) subnets.
In the cases where I know companies are trying to use Gerrit Code Review
internally, they want to position Gerrit as the border router, much like
they position Microsoft Exchange as the border router. All changes flow
through that router, with appropriate review controls and processes put
into place, so that they can ensure legal, IP, etc. have done their
parts necessary before they release code to the outside. Likewise with
bringing code in.
My life has lately been dominated by Gerrit and its users, so I'm
currently pre-disposed towards thinking about their problems, and their
required features. Any group I'm talking to is betting large on code
bases managed through Git and Gerrit Code Review, they want to
contribute fixes back, but they also want to protect their assets and
their networks. So dedicated border routers are "The Way". And their
investment is large enough (sometimes engineering staff into the
hundreds of full timers) that asking IT to create such a border server
is extremely worthwhile.
But yea, you're right, I forgot about the private LANs that many
companies run, where HTTP and SMTP are the only way out, and in some
cases, yes, the company's investment isn't enough to justify a dedicated
border router for say Eclipse.org related items.
Thanks.
|
|
|
| Re: Patch handling process and mylyn? [message #3621 is a reply to message #3587] |
Thu, 23 April 2009 10:19 |
| Eclipse User |
|
|
|
Originally posted by: alex_blewitt.nospam.yahoo.com
Shawn Pearce wrote:
> Alex Blewitt wrote:
>> I'm trying to think aloud about how it would work. Having multiple patches
>> generated from a (say) git format-patch seems to be the tricky thing; so
how
>> do you attach them to a bug? As multiple attachments? Or can we concatenate
>> them together into a multi-patch file somehow?
>
> Yes.
>
> git format-patch --stdout ... >foo.mbox
> git am foo.mbox
That's great. It sounds like the way forward then would be to allow Mylyn/Git
to generate the patches in .mbox format and attach to the bug. It'd still be
textual (so you could reply/quote via the current quote attachment) and still
give you the ability to accept bulk patches whilst maintaining history. Sounds
like the way forward to me.
Incidentally, Eclipse has the concept of a 'unified team format' for its CVS
tooling which allows you to create a patch in a standard format across
different projects in the Eclipse workspace. So as well as the
history-preserving patches, it might be good to have the unified patch format
as a way of uploading a simple change without history.
> How quickly I forget that I spent the last 5+ years at a Fortune 10
> company. Their network was isolated. We used Eclipse. But yea, you're
> right, we only had a brain-damaged HTTP proxy to the outside world.
Sad but true. When you have your own company, particularly if it's a startup
or fresh from University, you can do what you want. I know I did :-) But a big
difference between Eclipse and other open-source projects is that Eclipse
tends to have more commercially interested partners, and (in particular)
platforms like Eclipse RCP often have a lot of internal use that don't see the
outside world. Making it easy to consume code from that perspective is an
important part of that aspect, and as you note, network isolation can mean
that an HTTP proxy is the way to the outside world. (Not sure about the
brain-damaged part though; HTTP is about transferring hypertext, and there's
no reason that code can't be considered hypertext with its cross references to
other files and the like. Why brain damaged?)
> In the cases where I know companies are trying to use Gerrit Code Review
> internally, they want to position Gerrit as the border router, much like
> they position Microsoft Exchange as the border router. All changes flow
> through that router, with appropriate review controls and processes put
> into place, so that they can ensure legal, IP, etc. have done their
> parts necessary before they release code to the outside. Likewise with
> bringing code in.
I think for companies that are hooked on Git, and have an out-flow, you may be
right. But I don't think all companies fit that flow. You'd have to have a
significant investment in Git and probably existing code flows out to make
that worthwhile; and some of the lumbering Fortune 10 companies of which you
speak are still using CVS and think SVN will be The Next Big Thing. But bear
in mind that bugs and improvements can still come from someone who has access
to the source code and is able to submit a patch via a bugzilla HTTP
attachment; it shouldn't require additional tools.
> My life has lately been dominated by Gerrit and its users, so I'm
> currently pre-disposed towards thinking about their problems, and their
> required features. Any group I'm talking to is betting large on code
> bases managed through Git and Gerrit Code Review, they want to
> contribute fixes back, but they also want to protect their assets and
> their networks. So dedicated border routers are "The Way". And their
> investment is large enough (sometimes engineering staff into the
> hundreds of full timers) that asking IT to create such a border server
> is extremely worthwhile.
Right, there's going to be a small set of companies that fit that bill,
perhaps. Adobe is doing a lot of stuff with Eclipse/Cocoa; they might merit
setting up a Gerrit proxy were that being used. But there's a much larger set
of companies who won't need the expense of a Gerrit proxy. Not only that, but
the infrastructure demands at Eclipse are pretty tight and it might be
difficult to get a new system installed/maintained ... after all, Git will
have to supersede one of CVS/SVN (or both), and one might argue that keeping
all three around wouldn't be that difficult ...
> But yea, you're right, I forgot about the private LANs that many
> companies run, where HTTP and SMTP are the only way out, and in some
> cases, yes, the company's investment isn't enough to justify a dedicated
> border router for say Eclipse.org related items.
I'm all for making things easier if we can. But let's not end up with a
process that only works if you have a direct, unfiltered connection to the
internet.
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #570942 is a reply to message #3188] |
Mon, 20 April 2009 06:45 |
|
Alex Blewitt schrieb:
> So I'm trying to find out what the plan is should things move to egit;
> presumably, the developer discussions would move off the
> git@vger.kernel.org mailing list and onto the eclipse.egit-dev mailing
> list? And would patches start to flow via bug attachments instead?
That's the way given by the Eclipse development process. Patches need to
be captured in Bugzilla bugs. This helps in tracking the IP log using
Bugzilla keywords and flags.
Of course, Bugzilla could CC the mailing list and discussions could
happen right away.
> Secondly, would we be able to take advantage of the nature of DVCS to
> leverage applying patches in any way, like encouraging people to put
> patches up via (say) github and then pulling in, or would that cause more
> complications?
I think pull is very problematic because of IP reasons. You never know
what you pull until you really pulled it. A patch is better because it
can be reviewed and clarified upfront.
-Gunnar
--
Gunnar Wagenknecht
gunnar@wagenknecht.org
http://wagenknecht.org/
|
|
| |
| Re: Patch handling process and mylyn? [message #571016 is a reply to message #3255] |
Mon, 20 April 2009 19:30 |
|
David Green schrieb:
> My understanding is that the terms and conditions of using Eclipse
> Bugzilla ensure that attachments are IP-clean.
No, using Bugzilla only ensures that the Eclipse.org website terms and
conditions apply. This ensures that all attachments are contributed
under EPL. However, the submitter/contributor still needs to confirm
that he wrote all the code and is allowed to contribute it. Sometimes an
employer may need to confirm as well. Even then, for some contributions
CQs may need to be filedas well for the legal team to review the
submitted code.
-Gunnar
--
Gunnar Wagenknecht
gunnar@wagenknecht.org
http://wagenknecht.org/
|
|
|
| Re: Patch handling process and mylyn? [message #571035 is a reply to message #3188] |
Mon, 20 April 2009 20:16 |
Robin Rosenberg
Messages: 332
Registered: July 2009 |
Senior Member |
|
|
Alex Blewitt wrote:
> Having put a few patches together on Egit (and getting used to Git in
> general, so please excuse any noob mistakes I make ...) I wanted to bring
> up a difference between the way that patches are handled via the *git
> review process and Eclipse.
>
> Patches, as described in the SUBMITTING_PATCHES guide
> (http://repo.or.cz/w/egit.git?a=blob;f=SUBMITTING_PATCHES;hb=HEAD)
> involves emailing the patch to the current (2) maintainers, plus a Cc to
> the (shared) *git mailing list. At some point, once they move past a
> review, they presumably get folded upstream.
If either Shawn or I think the patches are good we apply, sign off them and
push to the official repository.
> Eclipse, on the other hand, tends to use mail for discussions but patches
> attached to bugs. Some projects even formalise it as part of the process
> for comitters, not just contributions.
Projects tend to use e they feel comfortable with. Sometimes
there is an explicit process. The Linux kernel has a process to ensure
IP cleanliness whereby author and maintainers sign off the patches.
> So I'm trying to find out what the plan is should things move to egit;
> presumably, the developer discussions would move off the
> git@vger.kernel.org mailing list and onto the eclipse.egit-dev mailing
> list? And would patches start to flow via bug attachments instead?
Good question. There is also the question of whether JGit can continue
to coexist under the same roof (with different licenses) or if we need
to split them and different homes, bug trackers. No decisions have been
made yet. In any case we'd move off the git mailing list.
Our current problem is that a mailing list is much better for discussing
patches than a bug tracker, but a bug tracker is much better for tracking
bugs that no one is actively working on. Hence we add a report to the
tracker for problems we find but probably won't fix right away, but
post fixes with or without bug reports, to the forum where lazy people can
read and comment without spending any time on mechanical processes like
copying and pasting.
Another problem we find as projects reach more contributors is that we find
them in twisted, environments with obstructed internet access, an
substandard e-mail (policy enforced) clients that mangle content.
> Secondly, would we be able to take advantage of the nature of DVCS to
> leverage applying patches in any way, like encouraging people to put
> patches up via (say) github and then pulling in, or would that cause more
> complications?
The reason we don't just pull and merge trees from contributors is that
we want to sign-off the patches to mark them as IP-clean. One could do
things differently. Mailing patches is very easy since it require only the
infrastructure that everybody has anyway.
Pulling complete branches is useful for getting quick access to code
for testing and for merging subsystems, i.e. branches where subsystem
maintainers have signed off each and every patch.
> For those new to Eclipse/Mylyn, I've blogged about how to set up Google
> Code from Mylyn at
> http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and am
> currently using this to track the issues I've raised/commented on. I'd be
> interested if any of the existing or interested Egit participants find
> this useful.
Is that setup different from what what is described in the Egit wiki?
-- robin
|
|
|
| Re: Patch handling process and mylyn? [message #571088 is a reply to message #3321] |
Tue, 21 April 2009 13:06 |
Alex Blewitt
Messages: 946
Registered: July 2009 |
Senior Member |
|
|
>Robin Rosenberg wrote:
>> Alex Blewitt wrote:
>> Eclipse, on the other hand, tends to use mail for discussions but patches
>> attached to bugs. Some projects even formalise it as part of the process
>> for comitters, not just contributions.
>
> Projects tend to use e they feel comfortable with. Sometimes
> there is an explicit process. The Linux kernel has a process to ensure
> IP cleanliness whereby author and maintainers sign off the patches.
Right, but my point of starting this thread is that there is a current
disconnect between how the e/jgit projects operate and the way that
projects hosted at Eclipse operate. There's probably more stringent
processes relating to IP since the Eclipse Foundation host the code and so
need to guarantee a blood-line to descendants.
>> And would patches start to flow via bug attachments instead?
>
> Good question. There is also the question of whether JGit can continue
> to coexist under the same roof (with different licenses) or if we need
> to split them and different homes, bug trackers. No decisions have been
> made yet. In any case we'd move off the git mailing list.
I'm sure part of the review/proposal process will cover that. Maybe it's
possible for two different licenses to co-exist under the same project;
certainly, one of the sign-off conditions is that each plug-in has its own
LICENSE file. So it's quite likely that the JGit would have a BSD-esque
license (EDL?) whilst the EGit would have EPL. Whether that would mean to
projects (eclipse.jgit) or whether they'd be two modules under the same
project (like http://www.eclipse.org/dsdp/tm/ which contains both) is
something for the Eclipse Foundation to comment on.
> Our current problem is that a mailing list is much better for discussing
> patches than a bug tracker
Is it much better because that's what you're used to, or are there real
use cases that you have in mind? After all, bug tracking can work for a
back-and-forth of discussion and patches, as seen (e.g.) here:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=272124
> But a bug tracker is much better for tracking
> bugs that no one is actively working on. Hence we add a report to the
> tracker for problems we find but probably won't fix right away, but
> post fixes with or without bug reports
This has a fairly significant downside, in that it forces everyone to be a
member of the mailing list in order to contribute or discuss. I've just
removed myself from the Git mailing list because there's way too much
noise to be useful to me personally - and with everyone mailing patches,
my inbox was filling up significantly faster just through membership of
this list. In addition, almost all of the mail was (to me) junk; the only
items I was interested in was feedback about the patches I was sending in.
And yet the SUBMITTING_PATCHES guide requires sending the patch to the git
mailing list to co-operate.
That's a significantly higher burden than any other Eclipse project; quite
aside from the Eclipse process that Gunnar alluded to (which requires
patches to go through bugzilla), it means that contributors must be
expected to bear the full brunt of the spam that comes in. For some
projects, especially perhaps where there's a hard-core set of developers,
it might be modus operandi, but it precludes anyone else from contribution
occasionally.
> to the forum where lazy people can read and comment without spending
> any time on mechanical processes like copying and pasting.
I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=272970 to suggest
that this capability could be built into mylyn, which it isn't at present.
But note that having the attachment can then provide other functionality
such as a web-based diff:
https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=diff
(Incidentally, this should probably be something supported in the back end
by the Git provider in order to encourage adoption of Git by other
projects)
> Another problem we find as projects reach more contributors is that we find
> them in twisted, environments with obstructed internet access, an
> substandard e-mail (policy enforced) clients that mangle content.
Indeed. I don't think I've yet managed to successfully send mail other
than by cut, paste and pipe to sendmail. And even then the spam filter at
vger bounced it back.
> Mailing patches is very easy since it require only the
> infrastructure that everybody has anyway.
True, but then everyone has web access too. And there are client
complicating factors, which you don't get with the web. Plus, some
organisations have policies specifically for mailing code (I know of some
organisations whose outgoing mail filters check for code originating from
inside and explicitly prohibit the mail in case there's internal code
leaks). Not to mention archiving of e-mails, which may be required for
regulatory purposes.
>> For those new to Eclipse/Mylyn, I've blogged about how to set up Google
>> Code from Mylyn at
>> http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and am
>> currently using this to track the issues I've raised/commented on. I'd be
>> interested if any of the existing or interested Egit participants find
>> this useful.
> Is that setup different from what what is described in the Egit wiki?
Which setup are you referring to? I could find no mention of mylyn on the
Egit wiki. Perhaps you could provide a URL to the page you mean?
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #571116 is a reply to message #3353] |
Tue, 21 April 2009 19:37 |
Shawn O. Pearce
Messages: 82
Registered: July 2009 |
Member |
|
|
Alex Blewitt wrote:
>> Robin Rosenberg wrote:
>
> Right, but my point of starting this thread is that there is a current
> disconnect between how the e/jgit projects operate and the way that
> projects hosted at Eclipse operate.
Yes. That disconnect has more to do with our heritage than just wanting
to be different for the sake of being different.
EGit started out in early 2006 as a project away from Eclipse, because
to be honest, I didn't know where to begin to propose a project for
hosting at Eclipse.org. And now that we are working on the process, its
simply way too much effort. I never would have proposed EGit to
Eclipse.org back in 2006. I advertised EGit on git@vger because other
Git users were there, and other frontends (QGit, gitk) were advertised
there every once in a while. We stuck to git@vger because people
sometimes mention the plugin, or post patches there.
>>> And would patches start to flow via bug attachments instead?
>> Our current problem is that a mailing list is much better for discussing
>> patches than a bug tracker
>
> Is it much better because that's what you're used to, or are there real
> use cases that you have in mind?
From what I have seen, doing code reviews of proposed patches falls
into three levels of convenience for the reviewer:
* Most Convenient: Gerrit Code Review; Reitveld; Google Mondrian
Patches can be commented inline directly in code. Drafts can be
written, and later emailed once polished.
Its easy to bounce around between files, or commits in the same series.
Its easy to see interdiffs between different versions of the same
proposed change.
Its easy to see your prior comments inline on the left pane, while
looking at how the patch author addressed them in the right pane.
Its easy to download and patch locally, to compile, execute, or view
with other tools.
For a contributor, its easy to upload a proposed change. Gerrit
supports git push directly over SSH. Reitveld has a command line Python
script. Google Mondrian has its own upload script (not that Mondrian
matters to anyone but Googlers, but its dead simple to use).
* Somewhat Convenient: Email
Its easy to reply. Your MUA quotes the code, so you can insert messages
inline.
Its easy to bounce around, or save drafts, just use your MUAs navigation
features. Sadly, I'm not as good with mutt as I should be for that to
really work well for me; I find Gerrit to be much easier.
Its easy to download and patch locally, given git-am.
Where I find email lacking is its difficult to compare the current
latest version of a change against my prior review comments, to remind
myself of what I said, and if the author addressed my concerns.
For a contributor, its easy to use git send-email, if you can tame that
beast. Once you get it configured right for your MTA, its easy to fire
off a 25 patch series to the maintainers.
* Least Convenient: Bug Tracker
Yea, sure, attach a patch file to a bug tracker. You can use comments
on the bug to have a conversation.
But there's no automated quoting. There's no easy way to draw attention
to a particular line code.
There's no easy way to interdiff the changes.
There's no easy way to download a long series (5+ commits). If each is
its own bug, you need to download one patch file per bug, then apply
them in order. Its easier in a MUA to save a chunk of emails to a
single mbox and shove it through git-am. Or in Gerrit to just issue
"git pull review refs/changes/nn/nnnn/1" to merge the entire thing.
For a contributor, its easy to attach one patch file, but at 2 patches,
I draw the line. I abhor patching Apache MINA SSHD because I need to
generate patch files and attach them to discrete bugs in their JIRA bug
tracking system. Its no less then 7 mouse clicks PER PATCH. I'll never
contribute more than a minor bug fix to MINA SSHD as a result. The
effort required to upload the F*@!!@* change is overshadowed by the
value the patch might bring to me.
So one thing I'm _really_ not looking forward to with the move to
Eclipse.org is the IP requirement that all changes not committed
directly to the repository go through Bugzilla. I think its going to be
a higher burden for contributors. I think its going to cause pain for
Robin and I with regards to reviewing code prior to it being committed.
I really like the fact that I can't just commit whatever I damn well
please to egit.git. I like having Robin sanity check my work before it
enters the repository. I'm not perfect, I know Robin has saved me from
myself on more than one occasion.
> This has a fairly significant downside, in that it forces everyone to be
> a member of the mailing list in order to contribute or discuss.
Not true. git@vger is open-posting. Anyone can post, without being
subscribed. The list has a Reply-To-All policy, so that people who are
not subscribed, but who are listed on the To/CC fields still receive the
thread.
> That's a significantly higher burden than any other Eclipse project;
Obtaining an Eclipse.org Bugzilla account, with a new unique
username/password pair, is a lot more effort than drive-by-patching to
git@vger.
> I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=272970 to suggest
> that this capability could be built into mylyn, which it isn't at
> present. But note that having the attachment can then provide other
> functionality such as a web-based diff:
>
> https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=diff
>
> (Incidentally, this should probably be something supported in the back
> end by the Git provider in order to encourage adoption of Git by other
> projects)
Yes. Look at Gerrit Code Review[1]. Its a JGit backed, web based code
review system. It uses Apache MINA SSHD to provide git fetch service
for clients to easily download proposed changes, and git push service to
upload new changes for review. Currently it depends upon C git to
produce diffs, but as soon as JGit has Good Enough(tm) diff support,
Gerrit will move to use it.
Gerrit may actually be favorable to Eclipse.org. Its entire APLv2 or
BSD (JSch and JGit). It can be configured to enforce contributors to
sign agreements prior to being able to post code. Those agreements can
be electronic click-through, or paper based forms that require a clerk
to ACK in the system upon receipt. As a result, as a user, you know in
advance that everything on the review server has been promised to be
IP-clean by the author, and that they have completed a sufficient
contribution agreement. Pulling code from there is therefore (in
theory) as safe as pulling code from the main repository.
[1] http://code.google.com/gerrit/
>> Another problem we find as projects reach more contributors is that we
>> find them in twisted, environments with obstructed internet access, an
>> substandard e-mail (policy enforced) clients that mangle content.
Yes. This is where Gerrit's requirement for SSH is actually a problem.
Its damn convenient to do "git push review HEAD:refs/for/master" in
order to propose a change, but if you are behind a firewall that blocks
outgoing encrypted SSH connections on port 29418, you are SOL.
And thus email is very suitable.
But, usually such braindamaged firewalls have unrestricted HTTP and
HTTPS on port 80 and 443 and will proxy to almost any site out there.
Half of them don't even try to do MITM attacks on the SSL traffic and
thus bypass any content filtering that they otherwise do on port 80.
HTTP is the new TCP. *sigh*
|
|
|
| Re: Patch handling process and mylyn? [message #571134 is a reply to message #3353] |
Tue, 21 April 2009 19:58 |
Robin Rosenberg
Messages: 332
Registered: July 2009 |
Senior Member |
|
|
Alex Blewitt wrote:
>>Robin Rosenberg wrote:
>>> Alex Blewitt wrote:
>>> Eclipse, on the other hand, tends to use mail for discussions but
>>> patches attached to bugs. Some projects even formalise it as part of the
>>> process for comitters, not just contributions.
>>
>> Projects tend to use e they feel comfortable with. Sometimes
>> there is an explicit process. The Linux kernel has a process to ensure
>> IP cleanliness whereby author and maintainers sign off the patches.
>
> Right, but my point of starting this thread is that there is a current
> disconnect between how the e/jgit projects operate and the way that
> projects hosted at Eclipse operate. There's probably more stringent
> processes relating to IP since the Eclipse Foundation host the code and so
> need to guarantee a blood-line to descendants.
>
>>> And would patches start to flow via bug attachments instead?
>>
>> Good question. There is also the question of whether JGit can continue
>> to coexist under the same roof (with different licenses) or if we need
>> to split them and different homes, bug trackers. No decisions have been
>> made yet. In any case we'd move off the git mailing list.
>
> I'm sure part of the review/proposal process will cover that. Maybe it's
> possible for two different licenses to co-exist under the same project;
> certainly, one of the sign-off conditions is that each plug-in has its own
> LICENSE file. So it's quite likely that the JGit would have a BSD-esque
> license (EDL?) whilst the EGit would have EPL. Whether that would mean to
> projects (eclipse.jgit) or whether they'd be two modules under the same
> project (like http://www.eclipse.org/dsdp/tm/ which contains both) is
> something for the Eclipse Foundation to comment on.
>
>> Our current problem is that a mailing list is much better for discussing
>> patches than a bug tracker
>
> Is it much better because that's what you're used to, or are there real
> use cases that you have in mind? After all, bug tracking can work for a
> back-and-forth of discussion and patches, as seen (e.g.) here:
>
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=272124
>
>> But a bug tracker is much better for tracking
>> bugs that no one is actively working on. Hence we add a report to the
>> tracker for problems we find but probably won't fix right away, but
>> post fixes with or without bug reports
>
> This has a fairly significant downside, in that it forces everyone to be a
> member of the mailing list in order to contribute or discuss. I've just
> removed myself from the Git mailing list because there's way too much
> noise to be useful to me personally - and with everyone mailing patches,
> my inbox was filling up significantly faster just through membership of
> this list. In addition, almost all of the mail was (to me) junk; the only
> items I was interested in was feedback about the patches I was sending in.
> And yet the SUBMITTING_PATCHES guide requires sending the patch to the git
> mailing list to co-operate.
It's correct that the Git ML is too high volume for widespread use specific
to EGit/JGit, but we started out that way with very few complaints. You
can submit patches to the list without being subscribed and by convention
people reply directly to all participants in a thread, so if you post
something you get all feedback.
> That's a significantly higher burden than any other Eclipse project; quite
> aside from the Eclipse process that Gunnar alluded to (which requires
> patches to go through bugzilla), it means that contributors must be
> expected to bear the full brunt of the spam that comes in. For some
> projects, especially perhaps where there's a hard-core set of developers,
> it might be modus operandi, but it precludes anyone else from contribution
> occasionally.
>
>> to the forum where lazy people can read and comment without spending
>> any time on mechanical processes like copying and pasting.
>
> I raised https://bugs.eclipse.org/bugs/show_bug.cgi?id=272970 to suggest
> that this capability could be built into mylyn, which it isn't at present.
> But note that having the attachment can then provide other functionality
> such as a web-based diff:
>
> https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=diff
>
> (Incidentally, this should probably be something supported in the back end
> by the Git provider in order to encourage adoption of Git by other
> projects)
>
>> Another problem we find as projects reach more contributors is that we
>> find them in twisted, environments with obstructed internet access, an
>> substandard e-mail (policy enforced) clients that mangle content.
>
> Indeed. I don't think I've yet managed to successfully send mail other
> than by cut, paste and pipe to sendmail. And even then the spam filter at
> vger bounced it back.
Hmmm.
>> Mailing patches is very easy since it require only the
>> infrastructure that everybody has anyway.
>
> True, but then everyone has web access too. And there are client
> complicating factors, which you don't get with the web. Plus, some
> organisations have policies specifically for mailing code (I know of some
> organisations whose outgoing mail filters check for code originating from
> inside and explicitly prohibit the mail in case there's internal code
> leaks). Not to mention archiving of e-mails, which may be required for
> regulatory purposes.
It's not the ultimate, it's been good because it didn't require any
infrastructure setup.
BTW, Now I saw something I didn't see before. The Eclipse
bugzilla has an Edit Attachment as comment, which looks like the exact
same thing. I haven't tried pressing Submit, though. If it works as
I think, I'll shut up...
Github also has line-level commenting, which looks nice and then there's
Gerrit2.
>>> For those new to Eclipse/Mylyn, I've blogged about how to set up Google
>>> Code from Mylyn at
>>> http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html and
>>> am currently using this to track the issues I've raised/commented on.
>>> I'd be interested if any of the existing or interested Egit participants
>>> find this useful.
>> Is that setup different from what what is described in the Egit wiki?
>
> Which setup are you referring to? I could find no mention of mylyn on the
> Egit wiki. Perhaps you could provide a URL to the page you mean?
"Featured Links" Configuring Mylin. Leading to
http://code.google.com/p/egit/wiki/ConfiguringMylyn The generice web
connector and hasn't been very useful to me since it hasn't been full
roundtrip. I think bugzilla will be more useful to us.
-- robin
|
|
|
| Re: Patch handling process and mylyn? [message #571223 is a reply to message #3420] |
Wed, 22 April 2009 11:49 |
Alex Blewitt
Messages: 946
Registered: July 2009 |
Senior Member |
|
|
Robin Rosenberg wrote:
> It's correct that the Git ML is too high volume for widespread use specific
> to EGit/JGit, but we started out that way with very few complaints. You
> can submit patches to the list without being subscribed and by convention
> people reply directly to all participants in a thread, so if you post
> something you get all feedback.
Good to know.
>>> Is that setup different from what what is described in the Egit wiki?
>>
>> Which setup are you referring to? I could find no mention of mylyn on the
>> Egit wiki. Perhaps you could provide a URL to the page you mean?
>
> "Featured Links" Configuring Mylin. Leading to
> http://code.google.com/p/egit/wiki/ConfiguringMylyn The generice web
> connector and hasn't been very useful to me since it hasn't been full
> roundtrip. I think bugzilla will be more useful to us.
Right, I was looking in the wrong wiki. The 'wiki' link on the main page
(http://code.google.com/p/egit/) takes you to a git.or.cz site
(http://git.or.cz/gitwiki/EclipsePlugin) which doesn't have any search
results returned for Mylyn, which is why I asked.
There are a couple of differences, yes:
1) It uses 'can=2' instead of 'can=1' which only downloads the open issues
2) It adds 'can' and 'search' parameters which you can change for each
individual query, instead of just having one. I wrote about the kinds of
search fields that you can use e.g. owner:shawn.pearce or
owner:jonas.fonseca priority:low.
You can change the (code.google.com) wiki to read:
Query request URL:
${serverUrl}/csv?can=${can}&colspec=ID+Status+Type+Owner +Summary&q=${search}
Query pattern:
^"({Id}[0-9]+?)","({Status}.*?)","({Type}.*?)","({Owner}.*?) ","({Description}.
*?)"$
Parameter:
can (value 2)
Parameter:
search (no value)
There's more explaination on my post about it.
http://alblue.blogspot.com/2009/04/google-code-and-mylyn-red ux.html
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #571261 is a reply to message #3387] |
Wed, 22 April 2009 11:49 |
Alex Blewitt
Messages: 946
Registered: July 2009 |
Senior Member |
|
|
Shawn Pearce wrote:
> Alex Blewitt wrote:
>>> Robin Rosenberg wrote:
>>
>> Right, but my point of starting this thread is that there is a current
>> disconnect between how the e/jgit projects operate and the way that
>> projects hosted at Eclipse operate.
>
> Yes. That disconnect has more to do with our heritage than just wanting
> to be different for the sake of being different.
OK, that's good to know.
> * Somewhat Convenient: Email
>
> Its easy to reply. Your MUA quotes the code, so you can insert messages
> inline.
>
> Its easy to bounce around, or save drafts, just use your MUAs navigation
> features. Sadly, I'm not as good with mutt as I should be for that to
> really work well for me; I find Gerrit to be much easier.
>
> Its easy to download and patch locally, given git-am.
A goal of EGit should be to make it as easy to apply patches via Mylyn
as it currently is with SVN and CVS. That's the modus operandi for Eclipse
projects, and would be a big win for adoption.
> Where I find email lacking is its difficult to compare the current
> latest version of a change against my prior review comments, to remind
> myself of what I said, and if the author addressed my concerns.
One of the benefits of an issue tracker is that you log all the comments with
the patch, and it's persisted. So unlike losing mail from months ago, you've
got all the history with the same item.
> * Least Convenient: Bug Tracker
>
> Yea, sure, attach a patch file to a bug tracker. You can use comments
> on the bug to have a conversation.
>
> But there's no automated quoting. There's no easy way to draw attention
> to a particular line code.
Perhaps you've not seen the 'quote attachment' feature of bugzilla?
If you view the attachment, you get an 'Edit attachment as comment' button:
https://bugs.eclipse.org/bugs/attachment.cgi?id=132092&a ction=edit
That will quote the entire attachment in exactly the same way as a 'reply'
would in your MUA.
> There's no easy way to download a long series (5+ commits). If each is
> its own bug, you need to download one patch file per bug, then apply
> them in order.
I think we can be intelligent here to try and get a good working practice.
Whilst it's true that there's generally one patch per bug in Eclipse, tools
like Mylyn can make applying those patches easier.
One big learning curve is going to be the small nature of patches, or rather,
the combination of patches that are generated when you're off from master.
Perhaps there are other ways of approaching this, like attaching a
zip-of-patches as an attachment to bugzilla? If a change is logically for one
item, then does it make sense to attach them to different bugs?
> For a contributor, its easy to attach one patch file, but at 2 patches,
> I draw the line. I abhor patching Apache MINA SSHD because I need to
> generate patch files and attach them to discrete bugs in their JIRA bug
> tracking system.
That sounds like an issue with the process, rather than the use of an issue
tracking system.
How can we make it easier? One idea might be to have a multi-patch file, which
contains all the patches but in a single file (compressed or concatenated).
After all, one of your cited examples is that 'git-am' makes it easy. What do
we need to do to 'Eclipse/Mylyn' to make it easy? That seems to be the key
question.
> So one thing I'm _really_ not looking forward to with the move to
> Eclipse.org is the IP requirement that all changes not committed
> directly to the repository go through Bugzilla. I think its going to be
> a higher burden for contributors. I think its going to cause pain for
> Robin and I with regards to reviewing code prior to it being committed.
I wouldn't have put it at a higher burden for contributors over any other
Eclipse project.
> I really like the fact that I can't just commit whatever I damn well
> please to egit.git. I like having Robin sanity check my work before it
> enters the repository. I'm not perfect, I know Robin has saved me from
> myself on more than one occasion.
I'll all for code reviews. But code reviews don't have to be based on mail -
in fact, the bugzilla has a 'reviewer' which can be set to flag someone as
having reviewed the patch.
> Yes. Look at Gerrit Code Review[1]. Its a JGit backed, web based code
> review system. It uses Apache MINA SSHD to provide git fetch service
> for clients to easily download proposed changes, and git push service to
> upload new changes for review. Currently it depends upon C git to
> produce diffs, but as soon as JGit has Good Enough(tm) diff support,
> Gerrit will move to use it.
> [1] http://code.google.com/gerrit/
Did you mean http://code.google.com/p/gerrit/ ?
I can see something like that being useful when there are many Git projects.
It might be difficult in getting support for running an additional tool by the
Eclipse webmasters though. I think there'd probably need to be a high
penetration of Git projects to make it worthwhile.
> Yes. This is where Gerrit's requirement for SSH is actually a problem.
> Its damn convenient to do "git push review HEAD:refs/for/master" in
> order to propose a change, but if you are behind a firewall that blocks
> outgoing encrypted SSH connections on port 29418, you are SOL.
Right. In fact, speaking as someone who is behind a firewall most of the time,
these kind of requirements are more of a burden than you might imagine. Bear
in mind that a lot of Eclipse support comes from member companies who do have
firewalls and only allow HTTP/S out.
> But, usually such braindamaged firewalls have unrestricted HTTP and
> HTTPS on port 80 and 443 and will proxy to almost any site out there.
> Half of them don't even try to do MITM attacks on the SSL traffic and
> thus bypass any content filtering that they otherwise do on port 80.
>
> HTTP is the new TCP. *sigh*
True. A lof of the perceived advantages of SVN was that you could access it
over HTTP and thus be able to use it from such sites.
http://alblue.blogspot.com/2007/11/eclipse-svn-over-https.ht ml
We should strive to make all processes work at a minimum over HTTP/S in order
to get the biggest win. Any procses that involves SSH over whatever port isn't
likely to be something that gets a great deal of support IMHO.
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #571281 is a reply to message #3488] |
Wed, 22 April 2009 15:24 |
Shawn O. Pearce
Messages: 82
Registered: July 2009 |
Member |
|
|
Alex Blewitt wrote:
> A goal of EGit should be to make it as easy to apply patches via Mylyn
> as it currently is with SVN and CVS. That's the modus operandi for Eclipse
> projects, and would be a big win for adoption.
Ok, I'll have to look at how Mylyn does this with SVN and CVS then.
Because I haven't looked at it before. Hell, I haven't used SVN except
by git-svn in years. And I haven't used CVS at all in more than 6 or 7
years.
Long-term, I should consider connecting Gerrit Code Review with Mylyn.
A number of folks who use Gerrit also use Eclipse.
>> But there's no automated quoting. There's no easy way to draw attention
>> to a particular line code.
>
> Perhaps you've not seen the 'quote attachment' feature of bugzilla?
No, I hadn't seen that before.
But editing text in a browser text widget. *sigh* Now I need to go
find a "vi" textarea replacement plugin for Firefox. Someone out there
must have written one.
I should have also mentioned Review Board yesterday in the "really easy"
category. I haven't used it myself, but people like it.
> One big learning curve is going to be the small nature of patches, or rather,
> the combination of patches that are generated when you're off from master.
> Perhaps there are other ways of approaching this, like attaching a
> zip-of-patches as an attachment to bugzilla? If a change is logically for one
> item, then does it make sense to attach them to different bugs?
Hmmph. One one hand you raise the 'quote attachment' feature of
bugzilla, and then on the other you suggest making the attachment a ZIP
archive? How would bugzilla quote that attachment for commenting?
>> For a contributor, its easy to attach one patch file, but at 2 patches,
>> I draw the line. I abhor patching Apache MINA SSHD because I need to
>> generate patch files and attach them to discrete bugs in their JIRA bug
>> tracking system.
>
> That sounds like an issue with the process, rather than the use of an issue
> tracking system.
Is the current Eclipse process really any better with Bugzilla?
Assuming the awesome CVS team provider and Mylyn are being used? For
non-committers?
>> So one thing I'm _really_ not looking forward to with the move to
>> Eclipse.org is the IP requirement that all changes not committed
>> directly to the repository go through Bugzilla. I think its going to be
>> a higher burden for contributors. I think its going to cause pain for
>> Robin and I with regards to reviewing code prior to it being committed.
>
> I wouldn't have put it at a higher burden for contributors over any other
> Eclipse project.
No, EGit's burden is the same for any other Eclipse project.
My point is, at present, it appears that the burden of *being* an
Eclipse project is higher than what we have enjoyed up until now from
the Git mailing list, and the Git tooling.
>> Yes. Look at Gerrit Code Review[1].
>
>> [1] http://code.google.com/gerrit/
>
> Did you mean http://code.google.com/p/gerrit/ ?
*sigh*. Yes.
> I can see something like that being useful when there are many Git projects.
> It might be difficult in getting support for running an additional tool by the
> Eclipse webmasters though. I think there'd probably need to be a high
> penetration of Git projects to make it worthwhile.
As you said, the penetration of Git projects needs to be enough to make
it worthwhile. The foundation webmasters have asked that Git replace an
existing VCS. If we replaced either CVS or SVN, I think the penetration
of projects is sufficient to justify Git on the server.
One of the requirements for that replacement to be successful is a good
team provider, aka EGit, that can be shipped with the platform, like the
CVS team provider is.
If Eclipse.org supports a Git repository, its going to need to support a
secure method of writing to that repository, one that can be reasonably
audited, like the current CVS over SSH.
At that point, you are talking about running Gitosis or a home-brewed
solution over SSH, like GitHub runs. Gerrit Code Review also provides
similar access control support. It can be used as a heavyweight
replacement for Gitosis, with the bonus of having an integrated code
review system.
So, the point I was trying to make was, if Eclipse.org does wind up
supporting Git, it will need to also explore Git server tools, at least
one of which includes what I consider to be a better code review
interface than Bugzilla's "quote attachment" feature. But, I am also
very biased.
>> Yes. This is where Gerrit's requirement for SSH is actually a problem.
>
> Right. In fact, speaking as someone who is behind a firewall most of the time,
> these kind of requirements are more of a burden than you might imagine. Bear
> in mind that a lot of Eclipse support comes from member companies who do have
> firewalls and only allow HTTP/S out.
Every time I hear that, I question the competency of the IT staff, or
the dedication of management to the contribution.
>> HTTP is the new TCP. *sigh*
>
> True. A lof of the perceived advantages of SVN was that you could access it
> over HTTP and thus be able to use it from such sites.
See. These sorts of companies block the CVS pserver port, because they
don't want code going in or out of the company. So SVN goes off and
embeds its protocol into HTTP, so that it can bypass any company's
firewall rules.
So now the IT staff is saying to management "we block those nasty code
sharing programs!" and meanwhile the engineers are running SVN over
HTTPS, defeating management's request that code not be shared.
If management actually blessed the staff consuming code and contributing
code back to eclipse.org, then IT should be able to write the firewall
rule to permit the access.
If IT can't, then either its because the management really isn't that
committed, or the IT staff can't write reasonable firewall rules that
balance the security of the internal network against the business needs
of the organization.
I'm sorry, but I find it hard to believe that a member company of
Eclipse, who is paying $20k-$250k/year just in membership dues, is so
uncommitted to Eclipse that they want to encourage their staff to punch
through their own firewalls by tunneling everything over HTTP.
> We should strive to make all processes work at a minimum over HTTP/S in order
> to get the biggest win. Any procses that involves SSH over whatever port isn't
> likely to be something that gets a great deal of support IMHO.
Unless Git suddenly grows support for a more efficient HTTP protocol for
both fetch and push, that's insane. And even then, I think its insane.
Embedding a data transfer protocol into HTTPS just to bypass
conservative firewalls is simply nuts.
|
|
|
| Re: Patch handling process and mylyn? [message #571350 is a reply to message #3521] |
Wed, 22 April 2009 23:48 |
Alex Blewitt
Messages: 946
Registered: July 2009 |
Senior Member |
|
|
Shawn Pearce wrote:
> Alex Blewitt wrote:
>> One big learning curve is going to be the small nature of patches, or
rather,
>> the combination of patches that are generated when you're off from master.
>> Perhaps there are other ways of approaching this, like attaching a
>> zip-of-patches as an attachment to bugzilla? If a change is logically for
one
>> item, then does it make sense to attach them to different bugs?
>
> Hmmph. One one hand you raise the 'quote attachment' feature of
> bugzilla, and then on the other you suggest making the attachment a ZIP
> archive? How would bugzilla quote that attachment for commenting?
I'm trying to think aloud about how it would work. Having multiple patches
generated from a (say) git format-patch seems to be the tricky thing; so how
do you attach them to a bug? As multiple attachments? Or can we concatenate
them together into a multi-patch file somehow? It was more a way of
investigating what would be the best way to replicate the ease of applying
many patches without having to have many attachments/bugs (which as noted
earlier, above 2 makes it detrimental)
> Is the current Eclipse process really any better with Bugzilla?
> Assuming the awesome CVS team provider and Mylyn are being used? For
> non-committers?
It's actually pretty easy, yes. I've submitted a number of patches to a number
of different projects by finding, fixing, then submitting the patch to the bug
and leaving it to go from there.
> My point is, at present, it appears that the burden of *being* an
> Eclipse project is higher than what we have enjoyed up until now from
> the Git mailing list, and the Git tooling.
Possibly - but the goal of EGit is to make it much easier to use. Perhaps
instead of the reliance on external (mail-based) tooling, we should focus on
tight integration with Mylyn. Actually, I suspect most of the hard work is
already there, since Mylyn is the one that talks to back ends already.
http://wiki.eclipse.org/Mylyn_FAQ#Team_Support
The key here is that it's the tooling that needs enhancement, rather than the
back end particularly. As a case in point, when creating a CVS diff in Mylyn,
it's possible to dump that patch to the clipboard and then have Mylyn upload
that patch from the clipboard without ever hitting the disk.
>>> Yes. This is where Gerrit's requirement for SSH is actually a problem.
>>
>> Right. In fact, speaking as someone who is behind a firewall most of the
time,
>> these kind of requirements are more of a burden than you might imagine.
Bear
>> in mind that a lot of Eclipse support comes from member companies who do
have
>> firewalls and only allow HTTP/S out.
>
> Every time I hear that, I question the competency of the IT staff, or
> the dedication of management to the contribution.
Every time I hear that, I wonder to what extent the involvement with large
companies is :-)
> See. These sorts of companies block the CVS pserver port, because they
> don't want code going in or out of the company. So SVN goes off and
> embeds its protocol into HTTP, so that it can bypass any company's
> firewall rules.
That's a pretty naive assumption of the world. It's not just a case of
particular ports being blocked (or opened) - more often than not, the internal
network is completely unreachable by any protocol/port to the outside world
unless it's in the blessed zone (which itself is firewalled in and out, too).
I haven't yet worked for a company in the last 10 years whose machines
internally had an IP connection to the outside world; they were all on private
(10.x.x.x) subnets.
Access to the outside world is via e.g. mail (and Outlook is not great if you
want to preserve niceities like spacing, frankly) and HTTP proxies, which
forward requests on your behalf. And even then, firewalls don't just let
anything go through on 80 or 443; in fact, connecting via a locked-down HTTPS
proxy will only let you CONNECT to other machines on 443 in some cases. A few
places have, for those who need it, SOCKS access to the outside world, which
is more of an IP proxy.
> So now the IT staff is saying to management "we block those nasty code
> sharing programs!" and meanwhile the engineers are running SVN over
> HTTPS, defeating management's request that code not be shared.
Shared != consumed != contributions, though. I've found several bugs in my
time from Eclipse whilst at work, but I can't investigate and find the bug
there and then because I can't get to HEAD. Often, it involves side-stepping
the process to download a TGZ from one of the nightly builds or other such
time and bandwidth wasters (for all sides).
> If management actually blessed the staff consuming code and contributing
> code back to eclipse.org, then IT should be able to write the firewall
> rule to permit the access.
Again, a private IP address isn't routable from the outside internet. It's not
just a firewall thing.
> If IT can't, then either its because the management really isn't that
> committed, or the IT staff can't write reasonable firewall rules that
> balance the security of the internal network against the business needs
> of the organization.
The business needs of the organisation are way more set up to prevent e.g.
viruses and access to machines from the outside than anything else. And as
noted, it's not just a firewall thing - these machines are unroutable from the
outside world and can only get there through proxies.
> I'm sorry, but I find it hard to believe that a member company of
> Eclipse, who is paying $20k-$250k/year just in membership dues, is so
> uncommitted to Eclipse that they want to encourage their staff to punch
> through their own firewalls by tunneling everything over HTTP.
Still unroutable machines from the outside world.
> Unless Git suddenly grows support for a more efficient HTTP protocol for
> both fetch and push, that's insane. And even then, I think its insane.
> Embedding a data transfer protocol into HTTPS just to bypass
> conservative firewalls is simply nuts.
OK, so that we're clear one last time - it's not about port blocking. It's
about having an unroutable machine inside a corporate network, communicating
via a proxy, in order to speak to the outside world. It's not a case of 'can't
you open up a port' or incompetence in the case of network engineers.
http://en.wikipedia.org/wiki/Private_network
"They are also commonly used in corporate networks, which for security
reasons, are not connected directly to the internet, meaning globally routable
addresses are unnecessary. Often a proxy, SOCKS gateway, or similar is used to
provide restricted internet access to internal users. In both cases, private
addresses are seen as adding security to the internal network, since it's
impossible for an Internet host to connect directly to an internal system."
Alex
|
|
|
| Re: Patch handling process and mylyn? [message #571375 is a reply to message #3554] |
Thu, 23 April 2009 00:58 |
Shawn O. Pearce
Messages: 82
Registered: July 2009 |
Member |
|
|
Alex Blewitt wrote:
> I'm trying to think aloud about how it would work. Having multiple patches
> generated from a (say) git format-patch seems to be the tricky thing; so how
> do you attach them to a bug? As multiple attachments? Or can we concatenate
> them together into a multi-patch file somehow?
Yes.
git format-patch --stdout ... >foo.mbox
git am foo.mbox
> Possibly - but the goal of EGit is to make it much easier to use. Perhaps
> instead of the reliance on external (mail-based) tooling, we should focus on
> tight integration with Mylyn. Actually, I suspect most of the hard work is
> already there, since Mylyn is the one that talks to back ends already.
Yes. Mik Kersten approach me about Mylyn during EclipseCon and wants to
help out with the integration effort. Connecting with the other tools
in the Eclipse environment is a great approach to making things
smoother. :-)
>> Every time I hear that, I question the competency of the IT staff, or
>> the dedication of management to the contribution.
>
> Every time I hear that, I wonder to what extent the involvement with large
> companies is :-)
:-)
How quickly I forget that I spent the last 5+ years at a Fortune 10
company. Their network was isolated. We used Eclipse. But yea, you're
right, we only had a brain-damaged HTTP proxy to the outside world.
I've been here at current job too long. Its a really big company.
Household name in pretty much any country. But we also really like the
Internet and I have pretty much unrestricted access from my desktop.
I'm spoiled. :-)
> It's not just a case of
> particular ports being blocked (or opened) - more often than not, the internal
> network is completely unreachable by any protocol/port to the outside world
> unless it's in the blessed zone (which itself is firewalled in and out, too).
> I haven't yet worked for a company in the last 10 years whose machines
> internally had an IP connection to the outside world; they were all on private
> (10.x.x.x) subnets.
In the cases where I know companies are trying to use Gerrit Code Review
internally, they want to position Gerrit as the border router, much like
they position Microsoft Exchange as the border router. All changes flow
through that router, with appropriate review controls and processes put
into place, so that they can ensure legal, IP, etc. have done their
parts necessary before they release code to the outside. Likewise with
bringing code in.
My life has lately been dominated by Gerrit and its users, so I'm
currently pre-disposed towards thinking about their problems, and their
required features. Any group I'm talking to is betting large on code
bases managed through Git and Gerrit Code Review, they want to
contribute fixes back, but they also want to protect their assets and
their networks. So dedicated border routers are "The Way". And their
investment is large enough (sometimes engineering staff into the
hundreds of full timers) that asking IT to create such a border server
is extremely worthwhile.
But yea, you're right, I forgot about the private LANs that many
companies run, where HTTP and SMTP are the only way out, and in some
cases, yes, the company's investment isn't enough to justify a dedicated
border router for say Eclipse.org related items.
Thanks.
|
|
|
| Re: Patch handling process and mylyn? [message #571408 is a reply to message #3587] |
Thu, 23 April 2009 10:19 |
Alex Blewitt
Messages: 946
Registered: July 2009 |
Senior Member |
|
|
Shawn Pearce wrote:
> Alex Blewitt wrote:
>> I'm trying to think aloud about how it would work. Having multiple patches
>> generated from a (say) git format-patch seems to be the tricky thing; so
how
>> do you attach them to a bug? As multiple attachments? Or can we concatenate
>> them together into a multi-patch file somehow?
>
> Yes.
>
> git format-patch --stdout ... >foo.mbox
> git am foo.mbox
That's great. It sounds like the way forward then would be to allow Mylyn/Git
to generate the patches in .mbox format and attach to the bug. It'd still be
textual (so you could reply/quote via the current quote attachment) and still
give you the ability to accept bulk patches whilst maintaining history. Sounds
like the way forward to me.
Incidentally, Eclipse has the concept of a 'unified team format' for its CVS
tooling which allows you to create a patch in a standard format across
different projects in the Eclipse workspace. So as well as the
history-preserving patches, it might be good to have the unified patch format
as a way of uploading a simple change without history.
> How quickly I forget that I spent the last 5+ years at a Fortune 10
> company. Their network was isolated. We used Eclipse. But yea, you're
> right, we only had a brain-damaged HTTP proxy to the outside world.
Sad but true. When you have your own company, particularly if it's a startup
or fresh from University, you can do what you want. I know I did :-) But a big
difference between Eclipse and other open-source projects is that Eclipse
tends to have more commercially interested partners, and (in particular)
platforms like Eclipse RCP often have a lot of internal use that don't see the
outside world. Making it easy to consume code from that perspective is an
important part of that aspect, and as you note, network isolation can mean
that an HTTP proxy is the way to the outside world. (Not sure about the
brain-damaged part though; HTTP is about transferring hypertext, and there's
no reason that code can't be considered hypertext with its cross references to
other files and the like. Why brain damaged?)
> In the cases where I know companies are trying to use Gerrit Code Review
> internally, they want to position Gerrit as the border router, much like
> they position Microsoft Exchange as the border router. All changes flow
> through that router, with appropriate review controls and processes put
> into place, so that they can ensure legal, IP, etc. have done their
> parts necessary before they release code to the outside. Likewise with
> bringing code in.
I think for companies that are hooked on Git, and have an out-flow, you may be
right. But I don't think all companies fit that flow. You'd have to have a
significant investment in Git and probably existing code flows out to make
that worthwhile; and some of the lumbering Fortune 10 companies of which you
speak are still using CVS and think SVN will be The Next Big Thing. But bear
in mind that bugs and improvements can still come from someone who has access
to the source code and is able to submit a patch via a bugzilla HTTP
attachment; it shouldn't require additional tools.
> My life has lately been dominated by Gerrit and its users, so I'm
> currently pre-disposed towards thinking about their problems, and their
> required features. Any group I'm talking to is betting large on code
> bases managed through Git and Gerrit Code Review, they want to
> contribute fixes back, but they also want to protect their assets and
> their networks. So dedicated border routers are "The Way". And their
> investment is large enough (sometimes engineering staff into the
> hundreds of full timers) that asking IT to create such a border server
> is extremely worthwhile.
Right, there's going to be a small set of companies that fit that bill,
perhaps. Adobe is doing a lot of stuff with Eclipse/Cocoa; they might merit
setting up a Gerrit proxy were that being used. But there's a much larger set
of companies who won't need the expense of a Gerrit proxy. Not only that, but
the infrastructure demands at Eclipse are pretty tight and it might be
difficult to get a new system installed/maintained ... after all, Git will
have to supersede one of CVS/SVN (or both), and one might argue that keeping
all three around wouldn't be that difficult ...
> But yea, you're right, I forgot about the private LANs that many
> companies run, where HTTP and SMTP are the only way out, and in some
> cases, yes, the company's investment isn't enough to justify a dedicated
> border router for say Eclipse.org related items.
I'm all for making things easier if we can. But let's not end up with a
process that only works if you have a direct, unfiltered connection to the
internet.
Alex
|
|
|
Goto Forum:
Current Time: Thu Apr 18 02:04:09 GMT 2024
Powered by FUDForum. Page generated in 0.04007 seconds |
] 
Search
Help
Register
Login
Home
