RE: [wtp-dev] Convention for "internal" packages

[David M Williams <david_williams@xxxxxxxxxx>]

I think you raise a lot of good points,
Konstantin, and it deserves serious consideration. 
I don't think we should make any wide
spread changes at this point in the release of WTP 2.0 ... 
but, there's always next year! And,
I wouldn't stand in the way if any particular component team 
decided to do it one way or the other,
as long as all in that component agreed, and it was well 
documented, and didn't break any existing
adopters ... as the JSF team "threatened" to do. 
Prime directive: break no adopters.


It is historically interesting ... it
was almost _exactly_ one year ago we were discussing this same 
topic on this same list. Apparently,
we somehow went from having 15,000 warnings to having 
50,000 in one year! 
http://dev.eclipse.org/mhonarc/lists/wtp-dev/msg03915.html
How'd that happen!? Rhetorical question
... there's actually only 662 I'm 
concerned about ... 100 of which happen
to be in the JSF component :)

"Konstantin Komissarchik" <kosta@xxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx

04/10/2007 10:18 PM

Please respond to
"General discussion of project-wide or architectural issues."        <wtp-dev@xxxxxxxxxxx>

To	"General discussion of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
cc
Subject	RE: [wtp-dev] Convention for "internal" packages

Reading the referenced wiki, it looks to me that these
rules are for the
platform SDK. With that in mind, I think this is something that should
be discussed at WTP level to see if the rules adopted by the platform
make sense in general and in the specific context of WTP. 

While the goal of not precludng experimentation is important, not
allowing plugins to have truly internal code can have very dangerous
consequences. For instance, it's very difficult to design sound api if
all your internals are hanging out in the open. This can be equated to
not allowing private modifiers on fields and methods. All it takes is
someone trying to enable their particular usecase by hitting internal
code in a way that it wasn't designed to handle and you can end up with
corrupted state for some other plugin to discover. Problems like this
are very difficult to understand and debug. The result is general lack
of stability.

Then you have the cases of people finding and using internal code when
perfectly good api is available. Sure the internal code is marked
x-internal, but people ignore that all the time. I had this very case
come up recently. There was an event object that I was exposing in an
api. It had to be down-cast to an event-specific type in order to get
additional detail. One adopter (showed up in adopter usage scans) chose
to cast it down to the impl class (in internal package with x-internal
tag) when a perfectly good interface was available in a public package.
If I didn't export the internal package, this wouldn't have happened.
The person writing that code would have looked a bit harder for the
interface and would not have cause wasted cycles for themselves and
others.

The above arguments are universal, but WTP has one that's specific to
us. The adopter usage scans and the adopter breakage agreement means
that the costs of experimentation with internal code are no longer
carried completely by the people doing the experimentation. Code that's
marked x-internal can show up in adopter breakage reports and cause
delays, extra overhead, and mitigation overhead for what should have
been a straight-forward change. Some of those usages are "justified"
(cases where there is no public api) some of them are not. We have no
way of telling the difference.

- Konstantin




-----Original Message-----
From: wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx]
On Behalf Of Cameron Bateman
Sent: Tuesday, April 10, 2007 4:21 PM
To: General discussion of project-wide or architectural issues.
Subject: Re: [wtp-dev] Convention for "internal" packages

Seems to me that if they really want to override our export rules, they
could simply import the relevent plugin as source and play around with
the code as they please.  For that matter, might it not be as simple
as
ignoring forbidden references? Or does the export-package manifest
actually cause things not be class loaded for external users?

In any case, we are still permitted to use x-friends from what I read
there.  This still allows me to create packages which, though exported,
are only visible to a narrow number of plugins.  Either I'm missing
something or this policy has not been well-thought out and tested.


--Cam

David M Williams wrote:

>
> > Is this policy recorded anywhere?  
>
> I think the best description of Export-Package policy is under 
> http://wiki.eclipse.org/index.php/API_Central
> which gives the reason as
>
> Not to mention if someone found it worth taking the risk to provide
a 
> better product! :)
>
> I know it's tempting to think we always know what's best for others
> .... but, imagine our indignation if the Eclipse Platform "refused"
to

> export those few cases of non-API which we in WTP have found to be
> essential.
>
> I think the root problem is we can not effectively use 'warnings'
> correctly, since
> we have too many (not to mention we in WTP are slow to evolve to API
> :) But, I think we can fix the Christmas tree effect ... over 50,000
> warnings in all!
>
> I'm all for discussion of the topic, but that's been our policy thus
far.
> And, I think it's the lessor of evils, IMHO.
> I honestly haven't surveyed or examined other projects in Eclipse
> (e.g.DTP, BIRT, TPTP) ... perhaps a question on cross-project list
> would be in order?
>
>
>
>
>
>
>
> *"Konstantin Komissarchik" <kosta@xxxxxxx>* Sent by:
> wtp-dev-bounces@xxxxxxxxxxx
>
> 04/10/2007 04:08 PM
> Please respond to
> "General discussion of project-wide or architectural issues."
      
>  <wtp-dev@xxxxxxxxxxx>
>
>
>                  
> To
>                  "General
discussion of project-wide or architectural issues." 
> <wtp-dev@xxxxxxxxxxx>
> cc
>                  
> Subject
>                  RE:
[wtp-dev] Convention for "internal" packages
>
>
>
>                  
>
>
>
>
>
> > Maybe it's just terminology, but it is Eclipse-wide policy that
ALL
> packages and
> > classes are made accessible (exported in the manifest.mf file).
>  
> Is this policy recorded anywhere? Seems backwards to me. What are
the 
> justifications for having it be this way. If you know that certain
> packages are really internal then they should not be exported.
> Exporting them (even if marked with x-internal) just invites people
to

> take a dependency on them causing both you and them grief down the
> road. I understand the usage of x-internal for the "provisional"
cases

> and the cases where there is no api, but forcing everything to be
> exported does not make any sense to me.
>  
> - Konstantin
>  
>
> ----------------------------------------------------------------------
> --
> *From:* wtp-dev-bounces@xxxxxxxxxxx
> [mailto:wtp-dev-bounces@xxxxxxxxxxx] *On Behalf Of *David M Williams*
> Sent:* Tuesday, April 10, 2007 12:52 PM*
> To:* General discussion of project-wide or architectural issues.*
> Subject:* Re: [wtp-dev] Convention for "internal" packages
>
>
> Yes, if you have one package that has both API and non-API Classes
in 
> it, then x-internal wouldn't help.
> I didn't realize that was the case you were talking about, so, 
> something has to be done there no matter what the new name would be,
> so just as well be 'internal').
>
> You raise some interesting points about x-friends vs. x-internal that
> I was not aware of.
> In fact, I thought it was the reverse ... that x-friends automatically

> meant it was x-internal to all (but the few named as friends). So,
I 
> learn something new everyday.
>
> But, I think x-friends is only appropriate for plugins within the
same

> component feature.
> http://www.eclipse.org/webtools/development/arch_and_design/subsystems
> /SubsystemsAndFeatures.html
>
> Guess we need to update that document to include JSF and JPA.
>
> Also, to correct something in Ian's note, and perhaps share an Eclipse

> policy that may not be known to all. Ian said, "... since 
> previously-used packages may no longer be accessible.".
> Maybe it's just terminology, but it is Eclipse-wide policy that ALL
> packages and classes are made accessible (exported in the manifest.mf
> file).  If you all were thinking of having a different policy,
then 
> that should be reviewed.
>
> The good news is ... I have a solution for the Christmas tree effect
> I've been preparing, and almost ready to send a note about!
>
>
>
>
>
> *Cameron Bateman <cameron.bateman@xxxxxxxxxx>*
> Sent by: wtp-dev-bounces@xxxxxxxxxxx
>
> 04/10/2007 02:31 PM
> Please respond to
> "General discussion of project-wide or architectural issues."
      
>  <wtp-dev@xxxxxxxxxxx>
>
>                  
> To
>                  "General
discussion of project-wide or architectural issues." 
> <wtp-dev@xxxxxxxxxxx>
> cc
>                  
> Subject
>                  Re:
[wtp-dev] Convention for "internal" packages
>
>
>
>
>                  
>
>
>
>
>
>
> Correct me if I'm wrong but x-internal applies at the package level,
not
> the class level.  How does one differentiate API from internal
classes
> if they are not separated into different packages by name?  And
if
this
> is the case, what would be better as the differentiator than
"internal"?
>
> Also, x-internal has several serious problems that I can see.  First,
it
> only marks something "discouraged" rather than "restricted".
 In JSF,
we
> turn off "discouraged" since otherwise our WTP dependencies
light up
our
> warnings log like a (yellow) Christmas tree.  However, we do
respect
> strictly the "forbidden" flag, which is what you get if
you use
> x-friends.  Second, x-friends is actually *overriden* when you
use
> x-internal.
>
>
> --Cam
>
> David M Williams wrote:
>
> >
> > My thoughts on this are that "internal" in package
names is
old-school
> > and no longer needed since OSGI and the eclipse extensions makes
it
> > not necessary. It would still be ok to do, for redundancy, but,
not
> > really required since we can use x-internal. When starting with
a
new
> > package at the beginning of a develop cycle, it is fine to use
> > 'internal' in the name, but I do not sure it is worth any risk
at
all
> > this late, since the same information can be conveyed and documented
> > using x-internal.
> >
> > I do think it's important to avoid 'provisional', if it is not
too
> > disruptive to your clients/adopters at this point in the 2.0
cycle.
In
> > theory, we (WTP) should have no more 'provisional'. That was
a
> > temporary thing, and
> > in hindsight, not that useful (and, more disruptive than expected).
> > From here on out, new functionality that is exposed for clients
should
> > be API, or not. We still need to 'evolve' the existing provisional,
> > but that'll be a long term process, going through proper review,
etc.
> >
> > I'd suggest opening a bugzilla to document details of your proposed
> > changes, and ideally provide changes to clients for review in
a
> > temporary branch, and get some voice from the community of adopters.
> > After all, in the "cost/benefit" trade-offs, it is
them that would
> > have to pay a cost now, for a potential benefit later.  That
is, at
> > this late in the cycle, we should not be making any changes _simply_
> > for naming convention purity. But, in the case of 'provisional',
it
is
> > likely a less expensive change to make now, than later.
> >
> > Thanks,
> >
> >
> >
> >
> >
> > *"Ian Trimble" <ian.trimble@xxxxxxxxxx>*
> > Sent by: wtp-dev-bounces@xxxxxxxxxxx
> >
> > 04/10/2007 12:51 PM
> > Please respond to
> > "ian.trimble@xxxxxxxxxx" <ian.trimble@xxxxxxxxxx>;
Please respond to
> > "General discussion of project-wide or architectural issues."
     
> >  <wtp-dev@xxxxxxxxxxx>
> >
> >
> >                  
> > To
> >                  "wtp-dev@xxxxxxxxxxx"
<wtp-dev@xxxxxxxxxxx>
> > cc
> >                  
> > Subject
> >                  [wtp-dev]
Convention for "internal" packages
> >
> >
> >
> >                  
> >
> >
> >
> >
> >
> > We're cleaning up our package names and declaring API in the
JSF
Tools
> > Project. We will be refactoring to remove "internal.provisional"
from
> > our package names. Also, we have inherited some code that currently
> > does not include "internal" in the package name but
we do not
consider
> > it API. Is it enough to manipulate the bundle manifest to mark
as
> > "x-internal" for these non-API packages, or should
we also be
> > injecting "internal" into non-API package names? What
is the
convention?
> >  
> > Thanks,
> >  - Ian (JSF Tools Project)
> >  
> > ------------------------------------------------------------
> > Ian Trimble
> > JDeveloper Group
> > Oracle Corporation Canada Inc.
> > Office: (250) 954-0837
> > Email: _ian.trimble@oracle.com_ <mailto:ian.trimble@xxxxxxxxxx>
> > Web: _http://www.oracle.com_ <http://www.oracle.com/>
> > ------------------------------------------------------------
> > This email may contain confidential and privileged material for
the
> > sole use of the intended recipient. Any review or distribution
by
> > others is strictly prohibited. If you are not the intended recipient
> > please contact the sender and delete all copies.
> >  _______________________________________________
> > wtp-dev mailing list
> > wtp-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/wtp-dev
> >
>
>-----------------------------------------------------------------------
-
> >
> >_______________________________________________
> >wtp-dev mailing list
> >wtp-dev@xxxxxxxxxxx
> >https://dev.eclipse.org/mailman/listinfo/wtp-dev
> >  
> >
>
> _______________________________________________
> wtp-dev mailing list
> wtp-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/wtp-dev
>
>
> Notice: This email message, together with any attachments, may contain

> information of BEA Systems, Inc., its subsidiaries and affiliated
> entities, that may be confidential, proprietary, copyrighted and/or
> legally privileged, and is intended solely for the use of the 
> individual or entity named in this message. If you are not the 
> intended recipient, and have received this message in error, please
> immediately return this by email and then delete 
> it._______________________________________________
> wtp-dev mailing list
> wtp-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/wtp-dev
>
>-----------------------------------------------------------------------
-
>
>_______________________________________________
>wtp-dev mailing list
>wtp-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/wtp-dev
>  
>

_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev

Notice:  This email message, together with any attachments, may contain
information  of  BEA Systems,  Inc.,  its subsidiaries
 and  affiliated entities,  that may be confidential,  proprietary,
 copyrighted  and/or legally privileged, and is intended solely
for the use of the individual or entity named in this message. If you are
not the intended recipient, and have received this message in error, please
immediately return this by email and then delete it.
_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev