Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jetty-users] Issue in migrating from standalone Jetty 10.x to Jetty 12 using EE8

Thanks Joakim for the detailed explanation.

The reason why I am passing so many parameters on the command line is, we deploy the same Jetty distribution in multiple deployments with certain attributes configurable for each deployment like:
1. JVM memory arguments, threadpool, buffer sizes etc. (calculated at runtime based on hardware)
2. Directory and file paths, start/stop ports
3. SSL parameters based on security provider configuration (e.g. SUN, BouncyCastle)

So, although they are configurable in ini files, it's convenient to pass them at JVM arguments at runtime instead of changing the files. However, for now, I have hardcoded most of these parameters in ini files as you mentioned and shortened the command line to see the --dry-run output. It didn't change much except for jvm.ini where it is forking a new JVM. So effectively, the order of execution of XMLs, jetty core classpath and EE8 classpath remained the same as I mentioned in previous email. 

But the error "java.lang.ClassNotFoundException: javax.servlet.http.HttpServletRequest" persists. I found the reason for it - which is due to a custom request logger implementation I have in my application. The line: 

<New id="RequestLog" class="org.eclipse.jetty.server.CustomRequestLog">

in my jetty\egainbase\etc\jetty-requestlog.xml looks like:

<New id="RequestLog" class="com.egain.jetty.requestlog.JettyRequestLogger">

Here JettyRequestLogger extends CustomRequestLog and logs some custom attributes in addition to few attributes in org.eclipse.jetty.server.Request object, which internally tries to load javax.servlet.http.HttpServletRequest class. Since this is a part of jetty-requestlog.xml, it is trying to find servlet API jar in core module classpath, which is failing for obvious reasons.

I would like to keep my custom implementation of request logger, but I have commented it out for now to move ahead. But then I stumbled upon another problem while deploying a war (which is well within the EE8 environment I believe). Here are the logs:

Dec 05, 2024 12:53:02 AM org.eclipse.jetty.server.Server doStart
INFO: jetty-12.0.15; built: 2024-11-05T19:44:57.623Z; git: 8281ae9740d4b4225e8166cc476bad237c70213a; jvm 21.0.2+13-58
Dec 05, 2024 12:53:02 AM org.eclipse.jetty.deploy.providers.ScanningAppProvider doStart
INFO: Deployment monitor ee8 in [file:///C:/work/jetty/jettybase/webapps/] at intervals 0s
Dec 05, 2024 12:53:02 AM org.eclipse.jetty.deploy.DeploymentManager addApp
INFO: addApp: App@225129c[ee8,null,C:\work\jetty\jettybase\webapps\admin.xml]
Dec 05, 2024 12:53:02 AM org.eclipse.jetty.ee8.annotations.AnnotationConfiguration lambda$getNonExcludedInitializers$1
INFO: Error:  for oeje8w.WebAppContext@4a3e3e8b{admin,/system/cb/admin,file:///C:/work/jetty/jettybase/work/21.20.0/admin.war/webapp/,false}{C:\work\jetty\jettybase/webapps/admin.war}
Dec 05, 2024 12:53:02 AM org.eclipse.jetty.ee8.annotations.AnnotationConfiguration lambda$getNonExcludedInitializers$1
INFO: Error:  for oeje8w.WebAppContext@4a3e3e8b{admin,/system/cb/admin,file:///C:/work/jetty/jettybase/work/21.20.0/admin.war/webapp/,false}{C:\work\jetty\jettybase/webapps/admin.war}
Dec 05, 2024 12:53:03 AM org.eclipse.jetty.ee8.webapp.WebAppContext doStart
WARNING: Failed startup of context oeje8w.WebAppContext@4a3e3e8b{admin,/system/cb/admin,file:///C:/work/jetty/jettybase/work/21.20.0/admin.war/webapp/,false}{C:\work\jetty\jettybase/webapps/admin.war}
java.lang.NoClassDefFoundError: javax/servlet/ServletContextListener
        at java.base/java.lang.ClassLoader.defineClass1(Native Method)
        at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1027)
        at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
        at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
        at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:580)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
        at org.eclipse.jetty.ee8.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:391)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
        at org.eclipse.jetty.util.Loader.loadClass(Loader.java:58)
        at org.eclipse.jetty.ee8.servlet.BaseHolder.doStart(BaseHolder.java:89)
        at org.eclipse.jetty.ee8.servlet.ListenerHolder.doStart(ListenerHolder.java:61)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.ee8.servlet.ServletContextHandler.startContext(ServletContextHandler.java:303)
        at org.eclipse.jetty.ee8.webapp.WebAppContext.startWebapp(WebAppContext.java:1195)
        at org.eclipse.jetty.ee8.webapp.WebAppContext.startContext(WebAppContext.java:1165)
        at org.eclipse.jetty.ee8.nested.ContextHandler.doStartInContext(ContextHandler.java:625)
        at org.eclipse.jetty.server.handler.ContextHandler$ScopedContext.call(ContextHandler.java:1446)
        at org.eclipse.jetty.ee8.nested.ContextHandler.doStart(ContextHandler.java:614)
        at org.eclipse.jetty.ee8.servlet.ServletContextHandler.doStart(ServletContextHandler.java:243)
        at org.eclipse.jetty.ee8.webapp.WebAppContext.doStart(WebAppContext.java:502)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
        at org.eclipse.jetty.server.Handler$Abstract.doStart(Handler.java:491)
        at org.eclipse.jetty.server.handler.ContextHandler.lambda$doStart$0(ContextHandler.java:754)
        at org.eclipse.jetty.server.handler.ContextHandler$ScopedContext.call(ContextHandler.java:1452)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:754)
        at org.eclipse.jetty.ee8.nested.ContextHandler$CoreContextHandler.doStart(ContextHandler.java:2271)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:40)
        at org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:183)
        at org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:522)
        at org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:162)
        at org.eclipse.jetty.deploy.providers.ScanningAppProvider.pathAdded(ScanningAppProvider.java:293)
        at org.eclipse.jetty.deploy.providers.ContextProvider.pathAdded(ContextProvider.java:639)
        at org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.pathAdded(ScanningAppProvider.java:69)
        at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:902)
        at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:868)
        at org.eclipse.jetty.util.Scanner.scan(Scanner.java:775)
        at org.eclipse.jetty.util.Scanner.startScanning(Scanner.java:664)
        at org.eclipse.jetty.util.Scanner.doStart(Scanner.java:636)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:120)
        at org.eclipse.jetty.deploy.providers.ScanningAppProvider.doStart(ScanningAppProvider.java:264)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.deploy.DeploymentManager.startAppProvider(DeploymentManager.java:592)
        at org.eclipse.jetty.deploy.DeploymentManager.doStart(DeploymentManager.java:263)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.server.Server.start(Server.java:641)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:120)
        at org.eclipse.jetty.server.Handler$Abstract.doStart(Handler.java:491)
        at org.eclipse.jetty.server.Server.doStart(Server.java:582)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:93)
        at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:2037)
Caused by: java.lang.ClassNotFoundException: javax.servlet.ServletContextListener
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
        ... 60 more

So the question is, why is it still complaining about Servlet API implementation classes? Note that if I copy jetty-servlet-api-4.0.6.jar and org.mortbay.jasper.apache-el-9.0.96.jar in C:\work\jetty\jettybase\lib\ext folder, everything works fine because I think these jars would then get included in jetty core classpath. But I am skeptical if this is the right way to do things, because in future I want to run apps in multiple environments like EE8 and EE10.

Appreciate your reply,
-Umesh.


On Tue, 3 Dec 2024 at 23:20, Joakim Erdfelt <joakim.erdfelt@xxxxxxxxx> wrote:
Your dry-run shows how Jetty operates.
Let's break it down, so you understand it.

# Java itself
'C:\work\jdk\bin\java.exe'
# Java specific System Properties
-Djava.io.tmpdir='C:\Users\devuser\AppData\Local\Temp\'
-Djetty.home='C:\work\jetty'
-Djetty.base='C:\work\jetty\jettybase'
# JVM specific ClassPath (using ClassPath mode, not Module mode)
# Notice that your --lib= argument shows up here, at the start of the classpath.
--class-path
'C:\work\eService\lib\egpl_classpath_mf.jar;C:\work\jetty\jettybase\resources;C:\work\jetty\lib\logging\slf4j-api-2.0.16.jar;C:\work\jetty\lib\logging\jetty-slf4j-impl-12.0.15.jar;C:\work\jetty\lib\jetty-alpn-client-12.0.15.jar;C:\work\jetty\lib\jetty-alpn-java-client-12.0.15.jar;C:\work\jetty\lib\jetty-client-12.0.15.jar;C:\work\jetty\lib\jetty-http-12.0.15.jar;C:\work\jetty\lib\jetty-server-12.0.15.jar;C:\work\jetty\lib\jetty-xml-12.0.15.jar;C:\work\jetty\lib\jetty-util-12.0.15.jar;C:\work\jetty\lib\jetty-io-12.0.15.jar;C:\work\jetty\lib\jetty-deploy-12.0.15.jar;C:\work\jetty\lib\jetty-session-12.0.15.jar;C:\work\jetty\lib\jetty-security-12.0.15.jar;C:\work\jetty\lib\jetty-ee-12.0.15.jar;C:\work\jetty\lib\jetty-plus-12.0.15.jar;C:\work\jetty\lib\jetty-jndi-12.0.15.jar'
# Java main-class to execute
org.eclipse.jetty.xml.XmlConfiguration
# Everything below here is used by XmlConfiguration.
# Properties that are used in XML
# These come from your ${jetty.base}/start.d/*.ini and any `[ini]` section in a ${jetty.home}/modules/*.mod that you have enabled.
java.version=21.0.2
jetty.base='C:\work\jetty\jettybase'
jetty.base.uri=file:///C:/work/jetty/jettybase
jetty.home='C:\work\jetty'
jetty.home.uri=file:///C:/work/jetty
jetty.httpConfig.headerCacheSize=4096
jetty.httpConfig.requestHeaderSize=16384
jetty.httpConfig.sendServerVersion=false
jetty.requestlog.dir=logs
jetty.session.evictionPolicy=420
jetty.ssl.idleTimeout=45000
jetty.ssl.sniHostCheck=false
jetty.ssl.stsIncludeSubdomains=true
jetty.ssl.stsMaxAgeSeconds=31536000
jetty.sslContext.keyManagerPassword
jetty.sslContext.keyStorePassword
jetty.sslContext.keyStoreType=PKCS12
jetty.sslContext.reload.scanInterval=3600
jetty.sslContext.trustStorePassword
jetty.sslContext.trustStoreType=PKCS12
jetty.webapp.addHiddenClasses=org.eclipse.jetty.logging.,file:///C:/work/jetty/lib/logging/,org.slf4j.
runtime.feature.alpn=true
slf4j.version=2.0.16 
# The list of XML to execute. Order of XML is important!
# These are executed to establish the core features of the Jetty server.
'C:\work\jetty\etc\jetty-bytebufferpool.xml'
'C:\work\jetty\etc\jetty-threadpool.xml'
'C:\work\jetty\etc\jetty.xml'
'C:\work\jetty\etc\jetty-deploy.xml'
'C:\work\jetty\etc\sessions\id-manager.xml'
'C:\work\jetty\etc\jetty-ee-webapp.xml'
'C:\work\jetty\etc\home-base-warning.xml'
'C:\work\jetty\jettybase\etc\jetty-http.xml'
'C:\work\jetty\etc\jetty-http-forwarded.xml'
'C:\work\jetty\etc\jetty-ssl.xml'
'C:\work\jetty\etc\jetty-ssl-context.xml'
'C:\work\jetty\etc\jetty-https.xml'
'C:\work\jetty\jettybase\etc\jetty-requestlog.xml'
'C:\work\jetty\etc\sessions\session-cache-hash.xml'
'C:\work\jetty\etc\jetty-ssl-context-reload.xml'
'C:\work\jetty\jettybase\etc\JettyServerIcon.xml'
# Beginning of Environment for label "ee8"
--env ee8 
# Establish URLClassLoader entries for Environment.
-cp 'C:\work\jetty\lib\jetty-servlet-api-4.0.6.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-nested-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-servlet-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-security-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-webapp-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-plus-12.0.15.jar'
-cp 'C:\work\jetty\lib\jakarta.transaction-api-1.3.3.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-annotations-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-analysis-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-commons-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-tree-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\jakarta.annotation-api-1.3.5.jar'
-cp 'C:\work\jetty\lib\ee8-apache-jsp\org.eclipse.jdt.ecj-3.38.0.jar'
-cp 'C:\work\jetty\lib\ee8-apache-jsp\org.mortbay.jasper.apache-el-9.0.96.jar'
-cp 'C:\work\jetty\lib\ee8-apache-jsp\org.mortbay.jasper.apache-jsp-9.0.96.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-apache-jsp-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-glassfish-jstl\jakarta.servlet.jsp.jstl.jakarta.servlet.jsp.jstl-api-1.2.7.jar'
-cp 'C:\work\jetty\lib\ee8-glassfish-jstl\org.glassfish.web.javax.servlet.jsp.jstl-1.2.5.jar'
-cp 'C:\work\jetty\lib\jetty-websocket-core-common-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-websocket-core-client-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-websocket-core-server-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-servlet-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-javax-websocket-api-1.1.2.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-javax-client-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-javax-common-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-javax-server-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-jetty-api-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-jetty-common-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-jetty-server-12.0.15.jar'
# Properties used by this Environment "ee8" specifically.
contextHandlerClass=org.eclipse.jetty.ee8.webapp.WebAppContext
eclipse.jdt.ecj.version=3.38.0
ee8.asm.version=9.7.1
ee8.jakarta.annotation.api.version=1.3.5
ee8.jakarta.servlet.jsp.jstl.api.version=1.2.7
ee8.javax.servlet.jsp.jstl.impl.version=1.2.5
ee8.jsp.impl.version=9.0.96
jetty.deploy.scanInterval=0 
# XML that are executed within the scope of the Environment "ee8"
# These XML operate under the classloader established by the Environment.
# If the XML references a class that doesn't exist in Environment "ee8" (or the parent classloader for core) then it's a ClassNotFoundException.
'C:\work\jetty\etc\jetty-ee8-webapp.xml'
'C:\work\jetty\etc\jetty-ee8-deploy.xml'

The original question you asked about "java.lang.ClassNotFoundException: javax.servlet.http.HttpServletRequest", tells me that the command line you are using isn't satisfying the needs of the "ee8" environment on your jetty-home/jetty-base setup.

If we look at the command line you pasted in the original question, but split up ....

# JVM bin
C:\work\jdk/bin/java
# Arguments that are better put into the `jvm` module's configuration at ${jetty.base}/start.d/jvm.ini
-javaagent:C:\work\eService\lib\ext\com.volkhart.memory\measurer.jar
-Xms3584m
-Xmx3584m
-server
# Remove, this is a --lib= argument on start.jar instead.
# Or put it into a ${jetty.base}/start.d/egain.ini with the line
# --lib=C:\work\eService\lib\egpl_classpath_mf.jar
-classpath C:\work\eService\lib\egpl_classpath_mf.jar
# More lines for ${jetty.base}/start.d/jvm.ini
-Xrs
-XX:+UseStringDeduplication
-Xss256K
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=C:\work\eService\logs\
-Dcom.sun.identity.systemconfiguration=com.egain.platform.framework.federation.saml2.FedletConfigManager
-XX:+DisableExplicitGC
-XX:MetaspaceSize=128m
-XX:MaxMetaspaceSize=512m
-XX:MaxTenuringThreshold=2
-XX:SurvivorRatio=10
-XX:MaxNewSize=768m
-XX:NewSize=768m
# This is really strange, most of these should be no-ops, doing nothing useful on JVM 17+.
# Seeing as you have even the internal ASM exposed, are you attempting to walk the classes
# with ASM or Reflect in some way that would need ALL of these?
--add-opens=java.base/sun.security.ssl=ALL-UNNAMED
--add-opens=java.base/sun.util.calendar=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/sun.security.x509=ALL-UNNAMED
--add-opens=java.xml/javax.xml.datatype=ALL-UNNAMED
--add-opens=java.xml/com.sun.org.apache.xerces.internal.jaxp.datatype=ALL-UNNAMED
--add-opens=java.xml/com.sun.org.apache.xml.internal.serialize=ALL-UNNAMED
--add-opens=java.sql/javax.sql=ALL-UNNAMED
--add-opens=java.logging/java.util.logging=ALL-UNNAMED
--add-opens=java.desktop/javax.imageio=ALL-UNNAMED
--add-opens=java.naming/javax.naming=ALL-UNNAMED
--add-opens=java.rmi/java.rmi=ALL-UNNAMED
--add-opens=java.management/java.lang.management=ALL-UNNAMED
--add-opens=java.scripting/javax.script=ALL-UNNAMED
--add-opens=java.prefs/java.util.prefs=ALL-UNNAMED
--add-opens=java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED
--add-opens=java.base/jdk.internal.org.objectweb.asm.util=ALL-UNNAMED
--add-opens=java.sql/java.sql=ALL-UNNAMED
# More things for ${jetty.base}/start.d/egain.ini
-DINSTALL_DIR=C:\work\eService
-DINSTALL_DIR_LOCAL=C:\work\eService
# More for ${jetty.base}/start.d/jvm.ini
-Djava.net.preferIPv4Stack=true
-Djava.security.policy=C:\work\eService\bin\platform\windows\egpl_system.policy
-Djava.rmi.activation.port=15099
-Deg.host.name=ussuhvin0135.egeng.info
-Djava.library.path=C:\work\eService\lib\dlls
-Deg.is.local.fs=false
-Djdk.tls.trustNameService=true
-Dorg.bouncycastle.rsa.allow_multi_use=true
-Djavax.net.ssl.trustStoreProvider=BCFIPS
-Djavax.net.ssl.trustStoreType=PKCS12
-Djavax.net.ssl.trustStorePassword=
-Djdk.tls.client.protocols=TLSv1.2
-Djdk.tls.server.protocols=TLSv1.2
-Dhttps.protocols=TLSv1.2
-Xlog:gc=info,phases=info,age=info,marking=info,ref=info:file=\"C:\work\eService\logs\eg_log_ussuhvin0135.egeng.info_printgc.log\":time,uptime:filecount=100,filesize=5m
-Dderby.system.home=C:\work\eService
-Dderby.stream.error.file=C:\work\eService\logs
-Dapp.http.port=9001
-Dapp.https.port=9001
# These belong in ${jetty.base}/start.d/ssl.ini
-Djetty.ssl.port=9001
# These belong in ${jetty.base}/start.d/ssl-context.ini
# These paths MUST be absolute paths.
-Djetty.sslContext.keyStorePath=../../../jks/ussuhvin0135.egeng.info.ks
-Djetty.sslContext.trustStorePath=../../jdk/lib/security/cacerts
-Djetty.sslContext.provider=BCJSSE
-Djetty.sslContext.keyStoreProvider=BCFIPS
-Djetty.sslContext.trustStoreProvider=BCFIPS
# This belongs in ${jetty.base}/start.d/jvm.ini
-Dhazelcast.io.selectorMode="selectwithfix"
# These are properties after start.jar or for XmlConfiguration, not JVM System Properties.
-DSTOP.PORT=15095
-DSTOP.KEY=egain_jetty
# More for ${jetty.base}/start.d/jvm.ini
-Dsun.net.http.allowRestrictedHeaders=true
# This property doesn't exist anywhere in Jetty 12
-Djetty.requestlog.loglatency=true
# You don't have the console-capture module enabled, this is an unused property.
-Djetty.console-capture.dir=C:\work\jetty/logs/
# This belongs in ${jetty.base}/start.d/requestlog.ini
# It MUST be an absolute path as well.
-Djetty.requestlog.filePath=../../eService/logs/eg_log_ussuhvin0135.egeng.info_yyyy_mm_dd.request.log
# This is a duplicate from above, and belongs in ${jetty.base}/start.d/http.ini
-Djetty.http.port=9001
# This is fine as a System Property, but doesn't really need to be defined.
# What's more important is that your PWD or CWD is the ${jetty.base} directory.
-Djetty.base=C:\work\jetty\jettybase
# These are not System Properties, but rather are Context (WebAppContext or ServletContextHandler) attributes
-Dorg.eclipse.jetty.server.Request.maxFormContentSize=2000000000
-Dorg.eclipse.jetty.server.Request.maxFormKeys=10000
# This belongs in ${jetty.base}/start.d/http-forwarded.ini
-Djetty.httpConfig.forwardedPortAsAuthority=false
# Put this in ${jetty.base}/start.d/egain.ini as a -D<key>=<value> entry
-Degain.processname=ApplicationServer
# These belong in ${jetty.base}/start.d/http.ini
-Djetty.http.acceptors=8
-Djetty.http.selectors=8
-Djetty.http.acceptQueueSize=250
# These belong in ${jetty.base}/start.d/threadpool.ini
-Djetty.threadPool.minThreads=10
-Djetty.threadPool.maxThreads=750
# This is the jar containing the main-class for bootstrap
-jar C:\work\jetty\start.jar
# Put this in ${jetty.base}/start.d/egain.ini
--lib=C:\work\eService\lib\egpl_classpath_mf.jar
# This only does start bootstrap debug, not Jetty runtime debug.
--debug
# If you actually had console-capture enabled, this wouldn't be needed.
# Also, if you DO have console-capure enabled, this MUST be removed (it's a console loop otherwise)
> out.txt 2>&1


If you follow this advice, your command line, when using start.jar, will be tiny.
Optionally, you can make these changes, and then use the output from --dry-run to run your environment in a clean way.

- Joakim


On Tue, Dec 3, 2024 at 8:58 AM umesh kudale <uhkudale@xxxxxxxxx> wrote:
Thanks Joakim for your input.

The overly complex command line is because it's a big application where many things are configurable. Most of these arguments are not specific to jetty. The effective command here is just:

 java -jar C:\work\jetty\start.jar --lib=C:\work\eService\lib\egpl_classpath_mf.jar

I haven't included any specific jetty xmls, inis and jars because they are configured while adding specific modules (along with ee8) and I believe it should pick up the default configuration while forming the command line. Please correct if my understanding is wrong.

Anyway, I tried multiple options of dry-run with a smaller set of modules, and then in my complex jetty command and it gives an output attached as dry-run.txt. As I understand, Jetty core is using following classpath:

--class-path 'C:\work\eService\lib\egpl_classpath_mf.jar;
C:\work\jetty\egainbase\resources;
C:\work\jetty\lib\logging\slf4j-api-2.0.16.jar;
C:\work\jetty\lib\logging\jetty-slf4j-impl-12.0.15.jar;
C:\work\jetty\lib\jetty-alpn-client-12.0.15.jar;
C:\work\jetty\lib\jetty-alpn-java-client-12.0.15.jar;
C:\work\jetty\lib\jetty-client-12.0.15.jar;
C:\work\jetty\lib\jetty-http-12.0.15.jar;
C:\work\jetty\lib\jetty-server-12.0.15.jar;
C:\work\jetty\lib\jetty-xml-12.0.15.jar;
C:\work\jetty\lib\jetty-util-12.0.15.jar;
C:\work\jetty\lib\jetty-io-12.0.15.jar;
C:\work\jetty\lib\jetty-deploy-12.0.15.jar;
C:\work\jetty\lib\jetty-session-12.0.15.jar;
C:\work\jetty\lib\jetty-security-12.0.15.jar;
C:\work\jetty\lib\jetty-ee-12.0.15.jar;
C:\work\jetty\lib\jetty-plus-12.0.15.jar;
C:\work\jetty\lib\jetty-jndi-12.0.15.jar

And EE8 environment is using this classpath:

-cp 'C:\work\jetty\lib\jetty-servlet-api-4.0.6.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-nested-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-servlet-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-security-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-webapp-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-plus-12.0.15.jar'
-cp 'C:\work\jetty\lib\jakarta.transaction-api-1.3.3.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-annotations-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-analysis-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-commons-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\asm-tree-9.7.1.jar'
-cp 'C:\work\jetty\lib\ee8-annotations\jakarta.annotation-api-1.3.5.jar'
-cp 'C:\work\jetty\lib\ee8-apache-jsp\org.eclipse.jdt.ecj-3.38.0.jar'
-cp 'C:\work\jetty\lib\ee8-apache-jsp\org.mortbay.jasper.apache-el-9.0.96.jar'
-cp 'C:\work\jetty\lib\ee8-apache-jsp\org.mortbay.jasper.apache-jsp-9.0.96.jar'
-cp 'C:\work\jetty\lib\jetty-ee8-apache-jsp-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-glassfish-jstl\jakarta.servlet.jsp.jstl.jakarta.servlet.jsp.jstl-api-1.2.7.jar'
-cp 'C:\work\jetty\lib\ee8-glassfish-jstl\org.glassfish.web.javax.servlet.jsp.jstl-1.2.5.jar'
-cp 'C:\work\jetty\lib\jetty-websocket-core-common-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-websocket-core-client-12.0.15.jar'
-cp 'C:\work\jetty\lib\jetty-websocket-core-server-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-servlet-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-javax-websocket-api-1.1.2.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-javax-client-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-javax-common-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-javax-server-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-jetty-api-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-jetty-common-12.0.15.jar'
-cp 'C:\work\jetty\lib\ee8-websocket\jetty-ee8-websocket-jetty-server-12.0.15.jar'

Here, if I move the jars in -cp option to C:\work\jetty\jettybase\lib\ext, application starts successfully. I know this is not the right way, but what's the right way to have this working?
Questions:
1. Why is Jetty core expecting servlet API dependencies?
2. Do I need to configure additional modules or include some additional command line arguments to make it working? Currently enabled modules are 'home-base-warning,server,ext,resources,ee8-jsp,ee8-deploy,ee8-jstl,ee8-websocket-javax, ee8-websocket-jetty,http,http-forwarded,requestlog,session-cache-hash,ssl,https,ssl-reload'.

Sorry, but I couldn't find any example for running Jetty 12 in standalone mode. Could you please give some more pointers?

On Mon, 2 Dec 2024 at 19:23, Joakim Erdfelt <joakim.erdfelt@xxxxxxxxx> wrote:
You are not using standalone correctly.

It seems you are instead creating a custom embedded Jetty setup.
Your command line ...
  • doesn't use the standalone core xml.
  • doesn't use the standalone core inis.
  • doesn't use the standalone core jars.
  • doesn't use the standalone configuration of environment jars (the ee8 classloader/classpath)
  • doesn't use the standalone configuration of environment xmls.
Don't just "port" Jetty 10 to Jetty 12, as-is.

Know that starting in Jetty 12, the server is split, core (with no Jakarta EE classes. eg: servlet).
And environments that provide Jakarta EE support (eg: ee8).
This is different from how it was in Jetty 10, where both core and servlet were intermingled.
Your command line would work fine in Jetty 10, but not in Jetty 12, as ee8 needs its own configuration (and classloader configuration).

Do this.
Forget about your overly complex command line. Ignore it for now.
Create a new (empty) jetty-base directory.
Properly configure it using the jetty-base/jetty-home configuration steps.
Enable a small subset, something like `ee8-deploy, ee8-webapp, ee8-annotations, http`
Then put your webapp into the `${jetty.base}/webapps/` directory.
Now, look at the `start.jar --list-config` output.
Next, look at the output of `start.jar --dry-run` (make sure to read the usage of --dry-run on `start.jar --help`)
Finally, look at how org.eclipse.jetty.util.component.Environment is built up from the command line during the normal standalone "main" class.

From there, you should be able to see what you need to do to get your custom solution working properly.

Good luck,
- Joakim

On Mon, Dec 2, 2024 at 2:07 AM umesh kudale via jetty-users <jetty-users@xxxxxxxxxxx> wrote:
I am currently migrating my application from standalone Jetty 10.x to Jetty 12.0.15. Since I am using Servlet 4.x, I have configured Jetty 12 with EE8 (for all the Jetty 10 modules I have been using so far, I have enabled corresponding EE8 modules). Here is the command I use to start Jetty:

C:\work\jdk/bin/java -javaagent:C:\work\eService\lib\ext\com.volkhart.memory\measurer.jar -Xms3584m -Xmx3584m -server -classpath C:\work\eService\lib\egpl_classpath_mf.jar -Xrs -XX:+UseStringDeduplication -Xss256K -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=C:\work\eService\logs\ -Dcom.sun.identity.systemconfiguration=com.egain.platform.framework.federation.saml2.FedletConfigManager -XX:+DisableExplicitGC -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=512m -XX:MaxTenuringThreshold=2 -XX:SurvivorRatio=10 -XX:MaxNewSize=768m -XX:NewSize=768m --add-opens=java.base/sun.security.ssl=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/sun.security.x509=ALL-UNNAMED --add-opens=java.xml/javax.xml.datatype=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xerces.internal.jaxp.datatype=ALL-UNNAMED --add-opens=java.xml/com.sun.org.apache.xml.internal.serialize=ALL-UNNAMED --add-opens=java.sql/javax.sql=ALL-UNNAMED --add-opens=java.logging/java.util.logging=ALL-UNNAMED --add-opens=java.desktop/javax.imageio=ALL-UNNAMED --add-opens=java.naming/javax.naming=ALL-UNNAMED --add-opens=java.rmi/java.rmi=ALL-UNNAMED --add-opens=java.management/java.lang.management=ALL-UNNAMED --add-opens=java.scripting/javax.script=ALL-UNNAMED --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED --add-opens=java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED --add-opens=java.base/jdk.internal.org.objectweb.asm.util=ALL-UNNAMED --add-opens=java.sql/java.sql=ALL-UNNAMED -DINSTALL_DIR=C:\work\eService -DINSTALL_DIR_LOCAL=C:\work\eService -Djava.net.preferIPv4Stack=true -Djava.security.policy=C:\work\eService\bin\platform\windows\egpl_system.policy -Djava.rmi.activation.port=15099 -Deg.host.name=ussuhvin0135.egeng.info -Djava.library.path=C:\work\eService\lib\dlls -Deg.is.local.fs=false -Djdk.tls.trustNameService=true -Dorg.bouncycastle.rsa.allow_multi_use=true -Djavax.net.ssl.trustStoreProvider=BCFIPS -Djavax.net.ssl.trustStoreType=PKCS12 -Djavax.net.ssl.trustStorePassword= -Djdk.tls.client.protocols=TLSv1.2 -Djdk.tls.server.protocols=TLSv1.2 -Dhttps.protocols=TLSv1.2 -Xlog:gc=info,phases=info,age=info,marking=info,ref=info:file=\"C:\work\eService\logs\eg_log_ussuhvin0135.egeng.info_printgc.log\":time,uptime:filecount=100,filesize=5m -Dderby.system.home=C:\work\eService -Dderby.stream.error.file=C:\work\eService\logs -Dapp.http.port=9001 -Dapp.https.port=9001 -Djetty.ssl.port=9001 -Djetty.sslContext.keyStorePath=../../../jks/ussuhvin0135.egeng.info.ks -Djetty.sslContext.trustStorePath=../../jdk/lib/security/cacerts -Djetty.sslContext.provider=BCJSSE -Djetty.sslContext.keyStoreProvider=BCFIPS -Djetty.sslContext.trustStoreProvider=BCFIPS -Dhazelcast.io.selectorMode="selectwithfix" -DSTOP.PORT=15095 -DSTOP.KEY=egain_jetty -Dsun.net.http.allowRestrictedHeaders=true -Djetty.requestlog.loglatency=true -Djetty.console-capture.dir=C:\work\jetty/logs/ -Djetty.requestlog.filePath=../../eService/logs/eg_log_ussuhvin0135.egeng.info_yyyy_mm_dd.request.log -Djetty.http.port=9001 -Djetty.base=C:\work\jetty\jettybase -Dorg.eclipse.jetty.server.Request.maxFormContentSize=2000000000 -Dorg.eclipse.jetty.server.Request.maxFormKeys=10000 -Djetty.httpConfig.forwardedPortAsAuthority=false -Degain.processname=ApplicationServer -Djetty.http.acceptors=8 -Djetty.http.selectors=8 -Djetty.http.acceptQueueSize=250 -Djetty.threadPool.minThreads=10 -Djetty.threadPool.maxThreads=750 -jar C:\work\jetty\start.jar --lib=C:\work\eService\lib\egpl_classpath_mf.jar --debug > out.txt 2>&1 

However, I am getting java.lang.ClassNotFoundException: javax.servlet.http.HttpServletRequest. The DEBUG output of the above command is attached as out.txt. I can see in the DEBUG output that the jetty servlet api jar is being included as -cp in the command line i.e. -cp C:\work\jetty\lib\jetty-servlet-api-4.0.6.jar. Why is it still giving these exceptions?
_______________________________________________
jetty-users mailing list
jetty-users@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/jetty-users

Back to the top