Configuring the parser component for a rules-based parser

The role of the parser is to take the messages that have been delimited by the extractor and build a set of mappings of string values to the Common Base Event attributes.

To configure the parser component, you need to configure rules for the two levels of parser execution:

Global processing

The global processing in the parser component takes a set of global regular expression rules and applies these rules against the message results provided by the extractor.

To configure the global processing rules for the parser component, follow these steps:

  1. In the left pane of the Adapter Configuration Editor, expand the node Configuration > Context Instance > Parser. The right pane displays the parser information.

    Parser Configuration details displayed in right pane

  2. Type the description of the parser in the Description field.
  3. If log records contain separator tokens that delimit the elements within a record, type the token in Separator Token field. For example, for the myapp application log record, the separator token between the elements is a double space: 
    <March 3, 2005 10:25:21 EST>  application:myapp  severity:3  message:WARNING-file style.css missing
    The separator token can be specified with the regular expression [ ]{2}.
  4. If the log records contain elements in a name-value pair pattern, you can specify the token that associates the name with the value in the Designation Token field. Using the same example as above, in the myapp log record, the designation token is a colon : .

Note: The values parsed during the global processing phase will be trimmed of leading and trailing white space characters. Therefore, substitution rules that use position values can be written assuming the values have no leading or trailing white space characters.

Attribute processing

The second phase of execution in the parser applies specific substitution rules to identify which values to assign to the Common Base Event attributes.

The following Common Base Event attributes are required:

For a complete listing of Common Base Event elements and their associated attributes, see Appendix I: Common Base Event format.

Adding and configuring elements and attributes

To add and configure an element:

  1. In the left pane of the Adapter Configuration Editor, select Configuration > Context Instance > Parser > Common Base Event. The right pane displays the Common Base Event information.
  2. To add an element, right-click on the Common Base Event node, and select the element that you want to add from the pop-up menu. The element is created beneath the Common Base Event node.
    In the example below, the situation element is added.

    The situation attribute is shown in the right pane

  3. To configure the element you just added, click on the element. The element properties will be displayed in the right pane.
  4. The Required by parent check box allows you to specify whether or not this element must have a valid value in order for the parent element to be valid. In the example above, if the check box is selected for the situation element, this indicates to the parser that the Common Base Event object must have a valid situation element. During processing, if the situation element returns a null value, the parser will not process any of the other child elements of the Common Base Event object since it already knows that this Common Base Event object is invalid because a required child element returned an invalid value.
  5. The Child Choice check box allows you to specify whether or not processing for this element should stop once a valid child attribute or element returns a valid value. For example, consider the situation element shown above. It has three child elements with their own subtrees. If the Child Choice check box is selected for the situation element, parsing will stop at its first child subtree that returns a valid value. This feature greatly improves the performance of parsing elements with multiple child elements or attributes.
    Note:categoryName is a required attribute of situation but the Generic Log Adapter will set categoryName to the appropriate value if one of the situation type elements returns a value.

To add and configure an attribute:

  1. In the left pane of the Adapter Configuration Editor, select Configuration > Context Instance > Parser > Common Base Event. The right pane displays the Common Base Event information.
  2. To add an attribute, right-click on the Common Base Event node, and select the attribute that you want to add from the pop-up menu. The attribute is created beneath the Common Base Event node.
    The example below shows the msg attribute added beneath the Common Base Event node.

    The msg attribute is shown in the right pane

  3. Click on the attribute to display the attribute properties on the right pane.
  4. Using the above example, if the log records do not all contain the msg attribute, you can select the Use Previous Match Substitution As Default check box. This will assign the value of the last matched message to the msg attribute.
  5. If you want to set a default value for the msg, type the value in the Default Value field. This default value will be used only if no substitution rules exist or match and the Use Previous Match Substitution As Default is not checked or if it is checked, there is no previous match value saved.
  6. The Required by parent check box for the attribute allows you to specify whether or not this attribute is required to have a valid value in order for the parent element to be valid.

Adding and configuring attribute substitution rules

To add a substitution rule to the Common Base Event attribute, select the Attribute > Add > Substitution Rule. The substitution rule details are displayed in the right pane of the editor.

  1. The Positions field can be used when the log record contains name value pairs delimited by a token. The value of a name value pair can be extracted using the expression $h('nameofNameValuePair').
    For example, using the same log record from myapp application: 
    <March 14, 2004 10:25:21 EST>  application:myapp  severity:3  message:WARNING-file style.css missing
    with the separator token = []{2} and the designation token = : , the expression to extract the message would be $h('message').
    For other applications of the Positions field, see the Positions field usage topic.
  2. The Match specifies a pattern to search for in the record. If the Positions field is specified, a part of the record will be searched, otherwise the whole record will be searched for the specified pattern. Within this pattern, certain data can be grouped together using parentheses which can be referred to in the Substitute field. If the pattern matches data in the record then the Common Base Event attribute for which the rule is written will be given a value based on the Substitute field. If the Match field is left empty, the rule will be considered a match.
    In the myapp log record example, to match the value of the message extracted by the positions field, the following regular expression can be used: 
    ^(\w+)-(.*)
    This match pattern contains two groups. The first group is a set of word characters before a dash. The second group is everything after the dash to the end of the message value as defined by the Positions field specification.
  3. The Substitute field is the value to be assigned to the Common Base Event attribute as a result of a match. The values can be extracted from the matched string using the literals $1, $2, etc., that represent the groups defined in the match pattern.
    In the myapp log record example, if you just want to assign the value of the warning to the Common Base Event msg attribute, specify the substitution as $1.

    Example of log record that is parsed and the result shown in the formatter results view

    For creationTime attributes, you can use the substitution keywords @YEAR, @MONTH, @DAY, and @TIMEZONE to use the corresponding Context properties Year, month, Day, and Timezone, respectively. For more information about these Context properties, see Configuring the adapter Contexts and Configuration for a rule-based adapter. For example, assume that a log record's time appears in the following format, and excludes the day, month, year, and time zone: 08:40:13.380. The following Substitute value adds the Day, Month, Year, and Timezone Context properties to the Common Base Event record:
    @DAY/@MONTH/@YEAR $1 @TIMEZONE
    Note, the above syntax assumes the Match property returns one grouping $1, which represents the time from the message.

  4. The Substitution Extension Class can be used if you want to run custom parsing logic on the value matched by the Positions, Match and Substitute fields. The user class must implement the Generic Log Adapter interface 
    org.eclipse.hyades.logging.adapter.parsers.ISubstitutionExtension
    in order for it to be executed by the Generic Log Adapter.

  5. The Time Format field can be used to create a substitution rule for the creationTime attribute. Specify a Java time pattern string in this field. The pattern entered will be used to parse the value matched by the Positions, Match and Substitute fields using the java.text.SimpleDateFormat class to generate the date as a long value. This value is then used to generate the string value of the date in the XML Schema dateTime format required by Common Base Event.
    In the myapp log record example, the Time Format rule will be as follows:

    Sample rule for the time format field using the Java simple date format class

  6. Apply the changes made by saving the adapter configuration file.
  7. To test your rules, click the Rerun Adapter Picture of the Rerun adapter icon to run the adapter. The results will be shown in the Formatter Result view. Hint: You can specify the number of the record that you want to start viewing from. Use the drop down available from the Re-run adapter icon and select Start from record. This will open up a dialog that allows you to specify a record number to start from.
  8. Click the Show Next Event Picture of the Show Next Event icon icon to view the Common Base Event attributes generated using the rules you have written.

Note: For more information on the Java simple date format, see the Javadoc HTML files java.sun.com/j2se/1.4.2/docs/api/java/text/SimpleDateFormat.html.

Once the parser rules are completed, you can configure the outputter component of the adapter configuration file.

Parent topic: Creating a rules-based adapter

Copyright © 2005 IBM Corporation. All rights reserved. This document and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which is available at http://www.eclipse.org/legal/epl-v10.html