Home » Eclipse Projects » EGit / JGit » Line endings again
| Line endings again [message #892065] |
Wed, 27 June 2012 04:44  |
|
Hi,
While many issues with EGit have got better over time there's one thing that really annoys me to death:
Almost constantly my staging view shows changed files that have no visible differences in the compare editor. Even
"Replace with version from Index" does not fix it. A hard reset does, though. And, even stranger, if I go to the Git
bash, say just "git status" and refresh the staging view in Eclipse then all the bogusmodifications go away. That can't
be right and it makes development impossible if there are hundreds of these non-dirty files in the staging view.
Is it possible that my configuration is still wrong? This is my System config file (from Windows 7 GitBash):
[core]
symlinks = false
autocrlf = true
[color]
diff = auto
status = auto
branch = auto
interactive = true
[pack]
packSizeLimit = 2g
[help]
format = html
[http]
sslCAinfo = /bin/curl-ca-bundle.crt
[sendemail]
smtpserver = /bin/msmtp.exe
[diff "astextplain"]
textconv = astextplain
[rebase]
autosquash = true
I think it makes no difference whether I clone with EGit or native Git. I always checkout with EGit, though.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| |
| Re: Line endings again [message #895802 is a reply to message #895798] |
Mon, 16 July 2012 06:20  |
 Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
Of course I mean Juno. Perhaps we can expect Kepler do address this?
Or hopefully something before another year goes by...
On 16/07/2012 8:04 AM, Ed Merks wrote:
> Eike,
>
> Perhaps someone from the git team will eventually comment on whether
> we should expect the Indigo version of Egit to handle line feed
> conversion properly. It's starting to seem unlikely though. It's an
> endlessly frustrating problem and to me would seem to be to be the
> most fundamental issue to be addressed with the highest possibly
> priority. Why that's not the case is a mystery to me.
>
>
> On 27/06/2012 6:44 AM, Eike Stepper wrote:
>> Hi,
>>
>> While many issues with EGit have got better over time there's one
>> thing that really annoys me to death:
>>
>> Almost constantly my staging view shows changed files that have no
>> visible differences in the compare editor. Even "Replace with version
>> from Index" does not fix it. A hard reset does, though. And, even
>> stranger, if I go to the Git bash, say just "git status" and refresh
>> the staging view in Eclipse then all the bogusmodifications go away.
>> That can't be right and it makes development impossible if there are
>> hundreds of these non-dirty files in the staging view.
>>
>> Is it possible that my configuration is still wrong? This is my
>> System config file (from Windows 7 GitBash):
>>
>> [core]
>> symlinks = false
>> autocrlf = true
>> [color]
>> diff = auto
>> status = auto
>> branch = auto
>> interactive = true
>> [pack]
>> packSizeLimit = 2g
>> [help]
>> format = html
>> [http]
>> sslCAinfo = /bin/curl-ca-bundle.crt
>> [sendemail]
>> smtpserver = /bin/msmtp.exe
>>
>> [diff "astextplain"]
>> textconv = astextplain
>> [rebase]
>> autosquash = true
>>
>> I think it makes no difference whether I clone with EGit or native
>> Git. I always checkout with EGit, though.
>>
>> Cheers
>> /Eike
>>
>> ----
>> http://www.esc-net.de
>> http://thegordian.blogspot.com
>> http://twitter.com/eikestepper
>>
>>
>
>
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| | |
| Re: Line endings again [message #895919 is a reply to message #895835] |
Mon, 16 July 2012 13:49 |
Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
Robin,
Unfortunately the general advice other Eclipse developers give me is
basically "don't use Egit, use the command line." You'll never get
feedback from folks like that. And of course when there are fundamental
problems that end up filling your staging view with noise, you'll never
get folks off of the command line in the first place to start giving
feedback. As I mentioned, many teams avoid the problem by using Unix
conventions on windows, but not all the tools that produce artifacts
respect such an approach; somehow one ends up with problems no matter
what strategy one takes.
In any case, to base your evaluation on feedback from patches or nightly
builds seems inadequate. I don't have time to patch together an Egit
and I need to focus on getting my own work done, not on evaluating
nightly builds that are potentially of poor quality and hence likely to
disrupt my work flow. I expect most people are in this position. We
need tools that work so that we can do our own work.
On 16/07/2012 11:04 AM, Robin Rosenberg wrote:
> Judging from feedback on patches and nightly builds, there seems to be
> very little interest in the CRLF handling, which is why the feature
> gets so little love.
>
>
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| |
| Re: Line endings again [message #896147 is a reply to message #896142] |
Tue, 17 July 2012 12:11 |
Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
Ondrej,
Comments below.
On 17/07/2012 2:04 PM, Ondrej Medek wrote:
> Hi Ed,
>
> as a workaround switch autocrlf = false and add a hook to your central
> repository, which will refuse any commits with CRLF line endings.
That doesn't help. There are generator tools that produce such
results. And there are tools on windows that expect windows conventions.
>
> Personally, I do not like this git autocrlf settings. It's the task of
> a text editor to set line endings.
It's the task of the version control system to extract results suitable
for the platform. Git, CVS, and SVN all do that nicely. Egit is the
exception to the rule.
> We have a bigger project (approx. 7 active developers), autocrlf =
> false and files have mixed line endings.
Yes, the version control system needs to manage that properly.
> Some files have LF only, some have CRLF, we do not care about it much.
Sounds like a mess. People ending up with the CRLF on unix systems will
often have problems. If you have a mixed team, some on windows and some
on mac, it also leads to problems.
> Eclipse handle this well - it keeps the line by default.
Yes, with EMF we try to be careful that the generators match the line
feed convention specified in the platform and, for existing files, the
convention in the file itself. But some generator tools run in their
own process without access to the workspace.
I still don't understand why the general advice ends up being don't use
the tool or work around the tool. Proper handling is fundamental for a
tool that deals with source files that are shared across OS platforms...
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: Line endings again [message #896246 is a reply to message #896147] |
Tue, 17 July 2012 17:13 |
R Shapiro
Messages: 386
Registered: June 2011 |
Senior Member |
|
|
Maybe we've just been lucky but core.autocrlf has been working fine for my team with egit 2.1. Of course for this to work, every developer must have the right configuration value ("true" in Windows, "input" in unix/linux), and has to have it configured at the time of the clone. If the origin repository is "dirty" (in the sense that it has DOS junk in it), probably worth cleaning it first. Otherwise you'll get some anomalies. Once the origin and all the clones are clean, they will remain that way. Works for us anyway.
As a result I'd much rather see the egit team fix other issues, for instance broken branch deletion from the "Branches" dialog in Juno, and even more important, the non-functional status decorations for local tracking. I hit these multiple times a day, every day. Git is all about "branchy" development, it's what distinguishes it from other distributed VC systems like Mercurial. These pieces really have to work right, and also have to be easy to access. Neither is true right now.
|
|
|
| Re: Line endings again [message #896322 is a reply to message #896246] |
Wed, 18 July 2012 04:49 |
|
Am 17.07.2012 19:13, schrieb R Shapiro:
> Maybe we've just been lucky but core.autocrlf has been working fine for my team with egit 2.1. Of course for this to
> work, every developer must have the right configuration value ("true" in Windows, "input" in unix/linux), and has to
> have it configured at the time of the clone. If the origin repository is "dirty" (in the sense that it has DOS junk
> in it), probably worth cleaning it first. Otherwise you'll get some anomalies. Once the origin and all the clones are
> clean, they will remain that way. Works for us anyway.
When you say "cleaning" do you mean rewriting the history or checking out all relevant branches, fixing the checkouts
and committing the results back?
Honestly, I don't see why EGit isn't doing the latter automatically.
> As a result I'd much rather see the egit team fix other issues, for instance broken branch deletion from the
> "Branches" dialog in Juno, and even more important, the non-functional status decorations for local tracking. I hit
> these multiple times a day, every day. Git is all about "branchy" development, it's what distinguishes it from other
> distributed VC systems like Mercurial. These pieces really have to work right, and also have to be easy to access.
> Neither is true right now.
Apart from the line ending hell the inability to do non-trivial merges is annoying me most. Even if I do the (pre-)
merge itself on the command line it's almost impossible to for me to use the compare editors to manually fix the
conflicts because (E)Git always just duplicates a file's content between these annoying conflict markers which is not
helpful at all. Maybe I'm doing something wrong ;-(
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| |
| Re: Line endings again [message #896455 is a reply to message #896427] |
Wed, 18 July 2012 12:28 |
Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Yes, Eike and I have been investigating the state of our repos and
it appears that it's indeed the case that we have DOS endings in the
repo itself. That leads to some of the problems we've been seeing
and we need to fix that. This type of git bash script (without the
test flags) looks promising<br>
<blockquote>find * -type f -exec dos2unix -b --test --verbose {} \;<br>
</blockquote>
We noticed too that files I commit with the latest git are properly
converted to Unix format in the repo. So that's a big improvement
and we're very happy with that!<br>
<br>
With a clean repo (with proper line endings in the actual repo) we
still see a significant problem. E.g., on Windows, if you clone
the repo for git://github.com/merks/sample.git, check out the
project, modify the README to add a line, and double click in the
staging view, you can see that all the lines are changed. If you
turn on "Show Whitespace Characters", you can see that the working
copy has DOS endings but the file in the index it's being compared
against has Unix endings. I would expect when autocrlf is true,
that the index file it's compared against will also have been
converted.<br>
<br>
That seems like a bug. Should I open a new bugzilla?<br>
<br>
<br>
<div class="moz-cite-prefix">On 18/07/2012 1:38 PM, R Shapiro wrote:<br>
</div>
<blockquote cite="mid:ju6786$1p0$1@xxxxxxxxe.org" type="cite">To
clarify "cleaning" (so to speak): Ideally everyone would have
core.autocrlf set properly before the first commit, and no
DOS-linebreak files would ever get into the origin repository in
the first place. This is fine for new repos. But you probably
have existing repos where the configuration was not initially
correct (or you were using an earlier version of egit that didn't
support autocrlf). In this case you need to do a pre-pass to
repair the existing commits in the origin. This is what I meant by
"cleaning", and what I had in mind was checkout/fix/commit, not
re-writing history. Possibly the latter is a better way to do
this, I don't know,
<br>
<br>
re: merge, I would have to agree that the Eclipse merge tool is
sub-optimal, but I don't think this is egit-specific -- it's about
the same in SVN/Subclipse. This is one case where IDEA does a
better job than Eclipse. Lately I've been playing around with
using the Perforce diff/merge tool, which is a free download and
can be integrated into Git/Egit. I'm still finding it a little
awkward but it has potential. I believe the integration procedure
was described some months ago on this forum. I can post my own
version here if anyone is interested.
<br>
</blockquote>
<br>
</body>
</html>
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: Line endings again [message #896485 is a reply to message #896455] |
Wed, 18 July 2012 13:57 |
R Shapiro
Messages: 386
Registered: June 2011 |
Senior Member |
|
|
Quote: E.g., on Windows, if you clone
the repo for git://github.com/merks/sample.git, check out the
project, modify the README to add a line, and double click in the
staging view, you can see that all the lines are changed. If you
turn on "Show Whitespace Characters", you can see that the working
copy has DOS endings but the file in the index it's being compared
against has Unix endings. I would expect when autocrlf is true,
that the index file it's compared against will also have been
converted
I don't run Windows myself so I can't test this but I'll see if I can find someone else here who can. If autocrlf is working, the README in the working directory should already have CRLF rather than LF -- this should happen as part of the checkout if your global configuration has core.autocrlf set to true, regardless of whether you make any modifications to the file. If that's not the case, something has already gone wrong. What version of egit are you using?
Note that if the cloning operation is also doing a checkout, as it normally would, you must have core.autocrlf set in your global config. Otherwise the checkout will be done with the wrong settings. If you don't want to set it globally you should therefore use the "--no-checkout" clone option, then set the repo config immediately after cloning, then checkout whatever branch you want. That's my understanding anyway.
I don't recall whether egit supports this clone option. You can probably also work around this problem by setting the repository config after the clone and then forcing a re-checkout.
As for which linebreaks are used in the index, that's an interesting question I hadn't considered. I guess I would expect to match the repository rather than the working directory. But I don't know the real answer.
|
|
|
| Re: Line endings again [message #896504 is a reply to message #896485] |
Wed, 18 July 2012 14:59 |
Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
Comments below.
On 18/07/2012 3:57 PM, R Shapiro wrote:
> Quote:
>> E.g., on Windows, if you clone
>> the repo for git://github.com/merks/sample.git, check out the
>> project, modify the README to add a line, and double click in the
>> staging view, you can see that all the lines are changed. If you
>> turn on "Show Whitespace Characters", you can see that the working
>> copy has DOS endings but the file in the index it's being compared
>> against has Unix endings. I would expect when autocrlf is true,
>> that the index file it's compared against will also have been
>> converted
>
>
>
> I don't run Windows myself so I can't test this but I'll see if I can
> find someone else here who can. If autocrlf is working, the README in
> the working directory should already have CRLF rather than LF -- this
> should happen as part of the checkout if your global configuration has
> core.autocrlf set to true, regardless of whether you make any
> modifications to the file.
Yes, that's definitely the case and is goodness!
> If that's not the case, something has already gone wrong. What
> version of egit are you using?
The version in Juno.
> Note that if the cloning operation is also doing a checkout, as it
> normally would, you must have core.autocrlf set in your global config.
Yes, I have that.
> Otherwise the checkout will be done with the wrong settings. If you
> don't want to set it globally you should therefore use the
> "--no-checkout" clone option, then set the repo config immediately
> after cloning, then checkout whatever branch you want. That's my
> understanding anyway.
>
> I don't recall whether egit supports this clone option. You can
> probably also work around this problem by setting the repository
> config after the clone and then forcing a re-checkout.
> As for which linebreaks are used in the index, that's an interesting
> question I hadn't considered. I guess I would expect to match the
> repository rather than the working directory. But I don't know the
> real answer.
In the end, the tool should only show me the lines I've changed, right?
In any case, I need to retest all this after fixing the line ending
problems in the repo itself...
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| | |
| Re: Line endings again [message #896599 is a reply to message #896552] |
Thu, 19 July 2012 05:34 |
Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Yes, all the steps you describe work well with Indigo. But your
steps didn't include what's not working well, i.e., double clicking
on the file in the staged view to see what the changes are. That's
where the problem is: the tool shows the entire file as follows:<br>
<blockquote><img src="http://www.eclipse.org/forums/index.php?t=getfile&id=10761" alt=""><br>
</blockquote>
See how the line endings are different and of course those are all
considered changes. Naturally, if you need to merge when Egit
thinks the entire file is changed, that will work poorly too, so
just hitting the ignore white space button isn't a complete
workaround, though it certainly helps.<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 18/07/2012 10:16 PM, R Shapiro
wrote:<br>
</div>
<blockquote cite="mid:ju75j7$sgm$1@xxxxxxxxe.org" type="cite">I
just did the following test:
<br>
<br>
- git clone git://github.com/merks/sample.git --no-checkout [I'm
doing no-checkout because my global core.autocrlf setting is
'input' and in for this test I want 'true']
<br>
- git config core.autocrlf true
<br>
- git checkout master
<br>
- Add the clone of Sample to my Git Repositories list in egit
<br>
- Import the project
<br>
- Open README and observe that the linebreaks are CRLF
<br>
- Edit README by adding a new line at the top
<br>
- Git Staging view shows one unstaged change: the added. This is
as expected.
<br>
- Undo the added line. modify one of the existing lines, and stage
the change
<br>
- Git Staging view shows one staged change: the added. This is as
expected.
<br>
- Commit the change. Everything looks good
<br>
<br>
So all this seems fine. The one obvious point of interest is that
I did the clone with command-line Git, which I had to do only
because my global aren't what I want for this special test case.
<br>
<br>
<br>
<br>
</blockquote>
<br>
</body>
</html>
Attachment: bjgaihbc.png
(Size: 27.84KB, Downloaded 532 times)
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: Line endings again [message #896600 is a reply to message #896599] |
Thu, 19 July 2012 05:49 |
Ed Merks
Messages: 33133
Registered: July 2009 |
Senior Member |
|
|
<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Sorry,<br>
<br>
It's early and I'm living in the past. I mean Juno again of
course. I spoke with Eike just now and he tried the nightly and it
too has this problem. We're continuing to investigate the behavior
of merge...<br>
<br>
I must say too that the quality of the tool has improved
dramatically and that we appreciate the help on this newsgroup; we
know from personal experience on the EMF newsgroup how much work it
is to provide newsgroup support. <br>
<br>
<br>
<div class="moz-cite-prefix">On 19/07/2012 7:34 AM, Ed Merks wrote:<br>
</div>
<blockquote cite="mid:ju8690$oai$1@xxxxxxxxe.org" type="cite">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
Yes, all the steps you describe work well with Indigo. But your
steps didn't include what's not working well, i.e., double
clicking on the file in the staged view to see what the changes
are. That's where the problem is: the tool shows the entire file
as follows:<br>
<blockquote><img src="cid:part1.03060200.05070808@xxxxxxxx"
alt=""><br>
</blockquote>
See how the line endings are different and of course those are all
considered changes. Naturally, if you need to merge when Egit
thinks the entire file is changed, that will work poorly too, so
just hitting the ignore white space button isn't a complete
workaround, though it certainly helps.<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 18/07/2012 10:16 PM, R Shapiro
wrote:<br>
</div>
<blockquote cite="mid:ju75j7$sgm$1@xxxxxxxxe.org" type="cite">I
just did the following test: <br>
<br>
- git clone git://github.com/merks/sample.git --no-checkout
[I'm doing no-checkout because my global core.autocrlf setting
is 'input' and in for this test I want 'true'] <br>
- git config core.autocrlf true <br>
- git checkout master <br>
- Add the clone of Sample to my Git Repositories list in egit <br>
- Import the project <br>
- Open README and observe that the linebreaks are CRLF <br>
- Edit README by adding a new line at the top <br>
- Git Staging view shows one unstaged change: the added. This
is as expected. <br>
- Undo the added line. modify one of the existing lines, and
stage the change <br>
- Git Staging view shows one staged change: the added. This is
as expected. <br>
- Commit the change. Everything looks good <br>
<br>
So all this seems fine. The one obvious point of interest is
that I did the clone with command-line Git, which I had to do
only because my global aren't what I want for this special test
case. <br>
<br>
<br>
<br>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
| |
Goto Forum:
Current Time: Thu Apr 18 07:12:53 GMT 2024
Powered by FUDForum. Page generated in 0.03423 seconds |
] 
Search
Help
Register
Login
Home
