Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[jdt-dev] Open Position: Compiler Apprentice

In the other thread we had some talk about the special character of the compiler part of JDT/Core and also about my current role in that.

I would love to specifically mentor one candidate who has ambitions to take some responsibility in this highly demanding, but also highly rewarding work.

Let me start by saying, that of course I'm not alone in that area:

Manoj and Jay work in all areas of JDT including the compiler. Unfortunately, much of their time is absorbed by implementing new features and requirements from new Java versions. That's why I speak of my responsibility for compiler *maintenance* - as that's the current division of labor between us.

I mentioned Till, who is very capable of fixing compiler bugs of any kind, but his total time for Eclipse is very limited (and his interest is not only on JDT/Core).

Many(!) others have provided the odd bug fix here or there. For those who weren't around: this was totally different when I became a JDT committer 10 years ago. NOBODY dropped in just to fix a bug or two. It was 100% IBM - until then :) Of course I'm more than happy how things have changed since then!

When I speak of an "apprentice" I'm thinking of helping you from one or two bug fixes towards mastering all areas of the compiler.

"Is this for me?"

It certainly helps, if you had contact to compilers before, perhaps you even remember a university class on this subject. You need to be prepared to dig into all details of JLS, check each i if it has a dot, and challenge every comma if it is an and or an or. The other arm of information is linked via git history. There's a wealth of bugs where JDT committers from all eras have discussed JLS, read between the lines of JLS and how that could possibly be the same language as what javac implements (it isn't). This information is indispensable, because only with this source you can understand why some code arms that contradict JLS are intended and necessary for some degree of javac compatibility - sigh.

On the positive side: once you have a test case for a given bug, there usually is no randomness, no intermittent failures (usually [1]). Still, the code location you need to debug may be hit thousands of times even on the simplest code example. So there is plenty of room for sharpening your debugging-fu.

Let's be honest: this course is not a matter of a few days, nor a few weeks. More likely we'll be speaking of years. Hopefully, this will be a gradual experience with more and more kinds of bugs surrendering when they see you. But if responsibility is the goal, then, yes, it's a matter of years.

"Is this really just about converting .java into .class?"

The compiler is the heart of more features than may be visible from the outside. I believe the most complex part is what they call "semantic analysis", split into resolving and flow analysis. This is where most errors and warnings are raised. At the far end, during code generation, there is one aspect that also I don't fully understand: generating stack maps. Olivier is the guru of that part. Perhaps also Olivier would like to pass on his specific knowledge, in particular after his recent re-work.

Right next to regular compilation there are two engines, which are essentially variants of the compiler, where the goal is not byte code, but resolving a word that was selected in the editor (SelectionEngine), or proposing completions (CompletionEngine). The latter two, are particularly sore spots, because the way they interact with the Parser is quite messy. More on that in a separate message.

Then there's a lot of converting between internal compiler AST and the other program representations Java Model and DOM AST, which then are the API through which clients (incl. JDT/UI & JDT/Debug) see Java programs.

Indexing & search all have their connections to (parts of) the compiler.

For an apprenticeship it might be interesting to ask Jay & Manoj if they can use a helping hand for a new Java feature, but note that these tasks have a tight and firm schedule. Otherwise, we have unbounded supply of bugs waiting to be analysed and fixed.

As your PhD you may even add an optional semantic analysis for detecting even more programming problems, and raising warnings.

Closer towards the point of taking responsibility, you'll be monitoring jdt-core-inbox and have lots of interesting discussions with users: (a) until they provide sufficient detail to reproducibly show their problem and (b) whether or not a difference between ecj & javac is our bug or "theirs". One of the nice comments I once received was: 'I knew it could also be a bug in javac, but I filed it at Eclipse because here I get timely answers' :).

Applications can be sent informally to this list :)


[1] I once and only once succeeded in introducing a concurrency bug into the compiler: which was one of the spookiest stories. Normally compiler bugs don't get 120 comments within a few weeks.

Back to the top