Home » Archived » Test and Performance Tools Platform (TPTP) » Some problem about Generic Log Adaptor .
| Some problem about Generic Log Adaptor . [message #21147] |
Mon, 04 July 2005 09:56  |
 wang xiao qing
Messages: 8
Registered: July 2009 |
Junior Member |
|
|
1) In the help contents,it said that "If the log file contains records that
span multiple lines in the file, select the Contains line breaks check box.
Do not select this check box if each line in the log file is a separate
record or if there are no line breaks in the log file". But if my log file's
record span multiple lines and at the same time does not contain any line
breaks,how can I deal with this type of record.Following(showed in blue
color) are two record examples,field in color Red stands for ClassName and
FunctionName,field in Dark-Green are all the debug message,which span
multiple lines.
[6/14/05 12:09:08:079 GMT+08:00] 20bef47f WC_RFQ d
YJacZzL8yKkezHKbs%2bqfYKB01AA%3d%0a RFQCreateCmdImpl.setRequestProperties
accessType = "0" /
name = UJIUVXQAJUQGRAN
ruletype = 1
jspStoreDir = store202
endtime = 0:00
enddate = 2005-12-30
endresult = 1
longdesc = MWLMNMGLIOXIRKISZKHO
URL = RFQModifyDisplay
shortdesc = ZNDPRBLQKJLBYBI
account_id = 100000
[6/14/05 12:09:08:094 GMT+08:00] 20bef47f WC_RFQ d
YJacZzL8yKkezHKbs%2bqfYKB01AA%3d%0a
com.ibm.commerce.utf.commands.RFQCreateCmdImpl.setRequestPro perties
startDate: null
where "d" means debug record which often span multiple lines in log
files.And at the same time,there is a null line after the multiple
spaning-record.I want to how can I read all of the spaned lines as one
record and how to treat these lines as a debug message field or
element,which designation token I should assign to( Here I define the
separator token as "[ ]{1,}",which means there is one or more null space
between each element.
2) In some extreme situations,there are some record lines stands for
Environment Information in the record lines which are not start with "[",how
can I deal with them.
************ Start Display Current Environment ************
WebSphere Platform 5.1 [JDK 1.4.2 ca1420-20040626] [BASE 5.1.1.3 cf30503.01]
running with process name svt2\svt2\WC_demo and process id 34364
Host Operating System is AIX, version 5.2
Java version = J2RE 1.4.2 IBM AIX build ca1420-20040626 (JIT enabled: jitc),
Java Compiler = jitc, Java VM name = Classic VM
was.install.root = /usr/WebSphere/AppServer
user.install.root = /usr/WebSphere/AppServer
Java Home = /usr/WebSphere/AppServer/java/bin/../jre
ws.ext.dirs =
/usr/WebSphere/AppServer/java/lib:/usr/WebSphere/AppServer/c lasses:/usr/WebSphere/AppServer/classes:/usr/WebSphere/AppSe rver/lib:/usr/WebSphere/AppServer/lib/ext:/usr/WebSphere/App Server/web/help:/usr/WebSphere/AppServer/deploytool/itp/plug ins/com.ibm.etools.ejbdeploy/runtime
Classpath =
/usr/WebSphere/AppServer/properties:/usr/WebSphere/AppServer /properties:/usr/WebSphere/AppServer/lib/bootstrap.jar:/usr/ WebSphere/AppServer/lib/j2ee.jar:/usr/WebSphere/AppServer/li b/lmproxy.jar:/usr/WebSphere/AppServer/lib/urlprotocols.jar
Java Library path =
/usr/WebSphere/AppServer/java/bin/../jre/bin:/usr/WebSphere/ AppServer/java/jre/bin/classic:/usr/WebSphere/AppServer/java /jre/bin:/usr/WebSphere/AppServer/bin:/usr/mqm/java/lib:/usr /opt/wemps/lib:/usr/local/staf/lib:/home/db2inst1/sqllib/lib :/usr/WebSphere/CommerceServer561/bin:/usr/WebSphere/AppServ er/bin:/usr/mqm/java/lib:/usr/opt/wemps/lib:/home/db2inst1/s qllib/lib:/usr/WebSphere/CommerceServer561/bin:/usr/lib
Current trace specification = com.ibm.websphere.commerce.WC_RFQ=all=enabled
************* End Display Current Environment *************
[6/14/05 12:04:11:133 GMT+08:00] 225ff47f ManagerAdmin I TRAS0017I: The
startup trace state is com.ibm.websphere.commerce.WC_RFQ=all=enabled.
The record showed above display some env. infomations(color blue) before the
record line(color red),I think it is hard to define common rule to extract
them.
3) Most of the record line have a field named EventType to define the format
of the record line.For example,"d" refer to a debug record ,"E" refer to a
exception record,"I" refer to a Enviroment Info and so on.Maybe different
type of eventtype have different numbers of fields,so I want to know how to
define common Parser rule to treat this situation.
If some one can give me a hand ,pls give me a reply soon! Thanks.
|
|
|
| Re: Some problem about Generic Log Adaptor . [message #24548 is a reply to message #21147] |
Tue, 02 August 2005 21:40  |
Eclipse User |
|
|
|
Originally posted by: nobody.nobody.com
Here are some suggestions for your questions:
1) If the log records always start with a timestamp enclosed in square
brackets, in the Extractor configuration you can specify a Start Pattern
that will match the timestamp. Since the timestamp will probably be
required to parse the creationTime value from the record you will want to
check the "Include start pattern" check box so the start pattern is included
in the record data to be parsed by the rules. If no End Pattern is
specified, the Extractor will determine the end of the record as the
character before the next Start Pattern match. Based on the records you
included in your append, here is a sample Start Pattern:
^\[\d{1,2}/\d{1,2}/\d{2,}\s\d{1,2}:\d{1,2}:\d{1,2}.*\]
To parse the data after the d character you can write a regex rule that
includes a d and have a group for all of the data following the d. For
example:
^\[\d{1,2}/\d{1,2}/\d{2,}\s\d{1,2}:\d{1,2}:\d{1,2}.*\]\s+\w+ \s\w+\s+d\s+(.*)
2) Yes you are correct it is sometimes difficult to deal with extraneous
header data in the rules. If that data appears at the top of the log file,
it will be bypassed by the Extractor if the Start Pattern is defined to look
for the timestamp within []. If the header data is included amongst other
records and the Extractor is configured as I suggested in 1) above, the
header data will be included at the end of the previous record and the
substitution rules need to account for that extra data. If there was an end
pattern defined for the log records, the header data could be ignored
because the Extractor would ignore any data between the end pattern of one
record and the start pattern of the next record. Here are a couple of GLA
feature enhancement requests that may help with this problem:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=92375
https://bugs.eclipse.org/bugs/show_bug.cgi?id=90866
3) I don't know of a way to write a common rule to cover all event types.
You probably need to write rules for each event type.
"wangxq" <xqwang@cn.ibm.com> wrote in message
news:dab13m$v2$1@news.eclipse.org...
> 1) In the help contents,it said that "If the log file contains records
that
> span multiple lines in the file, select the Contains line breaks check
box.
> Do not select this check box if each line in the log file is a separate
> record or if there are no line breaks in the log file". But if my log
file's
> record span multiple lines and at the same time does not contain any line
> breaks,how can I deal with this type of record.Following(showed in blue
> color) are two record examples,field in color Red stands for ClassName and
> FunctionName,field in Dark-Green are all the debug message,which span
> multiple lines.
>
> [6/14/05 12:09:08:079 GMT+08:00] 20bef47f WC_RFQ d
> YJacZzL8yKkezHKbs%2bqfYKB01AA%3d%0a RFQCreateCmdImpl.setRequestProperties
> accessType = "0" /
> name = UJIUVXQAJUQGRAN
> ruletype = 1
> jspStoreDir = store202
> endtime = 0:00
> enddate = 2005-12-30
> endresult = 1
> longdesc = MWLMNMGLIOXIRKISZKHO
> URL = RFQModifyDisplay
> shortdesc = ZNDPRBLQKJLBYBI
> account_id = 100000
>
> [6/14/05 12:09:08:094 GMT+08:00] 20bef47f WC_RFQ d
> YJacZzL8yKkezHKbs%2bqfYKB01AA%3d%0a
> com.ibm.commerce.utf.commands.RFQCreateCmdImpl.setRequestPro perties
> startDate: null
>
> where "d" means debug record which often span multiple lines in log
> files.And at the same time,there is a null line after the multiple
> spaning-record.I want to how can I read all of the spaned lines as one
> record and how to treat these lines as a debug message field or
> element,which designation token I should assign to( Here I define the
> separator token as "[ ]{1,}",which means there is one or more null space
> between each element.
>
> 2) In some extreme situations,there are some record lines stands for
> Environment Information in the record lines which are not start with
"[",how
> can I deal with them.
>
> ************ Start Display Current Environment ************
> WebSphere Platform 5.1 [JDK 1.4.2 ca1420-20040626] [BASE 5.1.1.3
cf30503.01]
> running with process name svt2\svt2\WC_demo and process id 34364
> Host Operating System is AIX, version 5.2
> Java version = J2RE 1.4.2 IBM AIX build ca1420-20040626 (JIT enabled:
jitc),
> Java Compiler = jitc, Java VM name = Classic VM
> was.install.root = /usr/WebSphere/AppServer
> user.install.root = /usr/WebSphere/AppServer
> Java Home = /usr/WebSphere/AppServer/java/bin/../jre
> ws.ext.dirs =
>
/usr/WebSphere/AppServer/java/lib:/usr/WebSphere/AppServer/c lasses:/usr/WebS
phere/AppServer/classes:/usr/WebSphere/AppServer/lib:/usr/We bSphere/AppServe
r/lib/ext:/usr/WebSphere/AppServer/web/help:/usr/WebSphere/A ppServer/deployt
ool/itp/plugins/com.ibm.etools.ejbdeploy/runtime
> Classpath =
>
/usr/WebSphere/AppServer/properties:/usr/WebSphere/AppServer /properties:/usr
/WebSphere/AppServer/lib/bootstrap.jar:/usr/WebSphere/AppSer ver/lib/j2ee.jar
:/usr/WebSphere/AppServer/lib/lmproxy.jar:/usr/WebSphere/App Server/lib/urlpr
otocols.jar
> Java Library path =
>
/usr/WebSphere/AppServer/java/bin/../jre/bin:/usr/WebSphere/ AppServer/java/j
re/bin/classic:/usr/WebSphere/AppServer/java/jre/bin:/usr/We bSphere/AppServe
r/bin:/usr/mqm/java/lib:/usr/opt/wemps/lib:/usr/local/staf/l ib:/home/db2inst
1/sqllib/lib:/usr/WebSphere/CommerceServer561/bin:/usr/WebSp here/AppServer/b
in:/usr/mqm/java/lib:/usr/opt/wemps/lib:/home/db2inst1/sqlli b/lib:/usr/WebSp
here/CommerceServer561/bin:/usr/lib
> Current trace specification =
com.ibm.websphere.commerce.WC_RFQ=all=enabled
> ************* End Display Current Environment *************
> [6/14/05 12:04:11:133 GMT+08:00] 225ff47f ManagerAdmin I TRAS0017I: The
> startup trace state is com.ibm.websphere.commerce.WC_RFQ=all=enabled.
>
> The record showed above display some env. infomations(color blue) before
the
> record line(color red),I think it is hard to define common rule to
extract
> them.
>
> 3) Most of the record line have a field named EventType to define the
format
> of the record line.For example,"d" refer to a debug record ,"E" refer to a
> exception record,"I" refer to a Enviroment Info and so on.Maybe different
> type of eventtype have different numbers of fields,so I want to know how
to
> define common Parser rule to treat this situation.
>
> If some one can give me a hand ,pls give me a reply soon! Thanks.
>
>
|
|
|
Goto Forum:
Current Time: Sat Apr 20 04:14:33 GMT 2024
Powered by FUDForum. Page generated in 0.02901 seconds |
] 
Search
Help
Register
Login
Home
