Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [eclipselink-dev] questions while running JPA JUnit tests 
Hi Tom,

> I have been running the tests and in general things are going well.

Great!

> I am concerned about one issue. There are a number of table creators 
> that have been altered with Symfoware specific code to switch to using a 
> FAST_TABLE_CREATOR. I am concerned that as people add new table creators 
> it will be hard to enforce including that kind of code for each table 
> creator.

Right.

> I have been trying to find a more generic way. I am attaching some rough 
> changes to TableCreator and SchemaManager.(the code still needs some work)
<snip>
> Can you think of any issues with making these kinds of changes?

No, it should be fine.

We just need to keep the code to clear the flag in cases such as
HistoricalEmployeeSystem.java (currently that's the only one):

        // reset useFastTableCreator flag as it might have been set by
tests that used
        // the non-historical table definitions
        creator.useFastTableCreator = false;

Thanks,
Dies


> Dies Koper wrote:
>> Hi Tom,
>>
>>> - EmployeeSystem - why are we removing the call to 
>>> dropTableConstraints()?
>>
>> I believe in the next line it calls 
>> EmployeeTableCreator().replaceTables(session), which calls 
>> super.replaceTables(session), which eventually calls 
>> replaceTablesAndConstraints, which has 
>> 'schemaManager.dropConstraints', so the constraints are dropped already.
>>
>> But if I misunderstood I can put it back in a 
>> SchemaManager.FAST_TABLE_CREATOR check block like the others. (or in a 
>> isSymfoware() block as Symfoware doesn't support foreign keys anyway.
>>
>>> - There are a number of places where a DeleteAll query has been replaces
>>> by a native query. (e.g. EntityMappingsAdvancedTestSuite,
>>> EntityManagerTLRJUnitTestSuite, EntityManagerJunitTestSuite) I am
>>> concerned that this does not account for the multi-table nature of
>>> Employee. I am also concerned that adding more native statements will
>>> not adapt if the model changes either to remove the multi-table nature,
>>> or to add another table) Perhaps we should isolate these calls to the
>>> case where Symfoware is active.
>>
>> Sure.
>> Would you like to skip the ones you are concerned about so I can pick 
>> them up easily after you've applied the rest?
>> Then I can prepare a new patch where I do the native queries in a 
>> isSymfoware() block (and double-check for missing queries on secondary 
>> tables).
>>
>> Thanks,
>> Dies
>>
>>
>>> Dies Koper wrote:
>>>> Hi Tom,
>>>>
>>>> I'm getting the same! Sorry about that.
>>>>
>>>> I've prepared the patch again, confirmed there is no error when
>>>> applying it, and uploaded it again.
>>>>
>>>> Thanks,
>>>> Dies
>>>>
>>>>
>>>> On 27/01/2010 05:55, Tom Ware wrote:
>>>>> Hi Dies,
>>>>>
>>>>> Can you try to reupload? I am getting an error that says: "An unknown
>>>>> line type was found in line 1380" when I try to apply the patch.
>>>>>
>>>>> -Tom
>>>>>
>>>>> Dies Koper wrote:
>>>>>> Hi Tom,
>>>>>>
>>>>>> I have attached the patch with the complete set of changes.
>>>>>> Please let me know if you have any questions.
>>>>>>
>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=288715
>>>>>>
>>>>>> Thanks,
>>>>>> Dies
>>>>>>
>>>>>>
>>>>>> On 26/01/2010 02:06, Tom Ware wrote:
>>>>>>> Hi Dies,
>>>>>>>
>>>>>>> Is it possible to provide a single patch with your complete 
>>>>>>> change set
>>>>>>> based on the latest code? If you can provide that, I'll start 
>>>>>>> working
>>>>>>> integrating and testing the changes.
>>>>>>>
>>>>>>> -Tom
>>>>>>>
>>>>>>> Dies Koper wrote:
>>>>>>>> Hi Tom, James,
>>>>>>>>
>>>>>>>> I ended up implementing this after all.
>>>>>>>> Patch (completely split settings) attached to the Symfoware issue.
>>>>>>>>
>>>>>>>> The reason is that Symfoware ignores the setMaxRows setting 
>>>>>>>> depending
>>>>>>>> on the cursor type, causing some tests to fail. At least with 
>>>>>>>> LIMIT I
>>>>>>>> can be sure that it works no matter what the cursor policy is.
>>>>>>>>
>>>>>>>> With this patch, Andrei's (for SCHEMA table name) and the previous
>>>>>>>> one
>>>>>>>> I uploaded, I have completed all changes to common code that I need
>>>>>>>> for Symfoware to work.
>>>>>>>> With these I passed 100% of the SRG test sets, and Core LRG.
>>>>>>>> Please let me know when you can accept the final patch with the
>>>>>>>> Symfoware platform class itself, the isSymfoware() methods and the
>>>>>>>> inclusions/exclusions to the tests for Symfoware.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Dies
>>>>>>>>
>>>>>>>>
>>>>>>>> On 16/12/2009 01:51, Tom Ware wrote:
>>>>>>>>> Hi Dies,
>>>>>>>>>
>>>>>>>>> To check this in, the setting will have to be completely split. 
>>>>>>>>> i.e.
>>>>>>>>> remove the methods that affect both settings
>>>>>>>>> (setIgnoreFirstResultsMaxRowsSettings and
>>>>>>>>> shouldIgnoreFirstResultsMaxRowsSettings). Instead, have separate
>>>>>>>>> getters
>>>>>>>>> and setters for each setting, and make the appropriate changes to
>>>>>>>>> classes depending on those methods.
>>>>>>>>>
>>>>>>>>> -Tom
>>>>>>>>>
>>>>>>>>> Dies Koper wrote:
>>>>>>>>>> Hi Tom,
>>>>>>>>>>
>>>>>>>>>>> I have started looking at your patch. I have a quick question:
>>>>>>>>>>>
>>>>>>>>>>> - In DatabaseAccessor, what is the goal of splitting the
>>>>>>>>>>> ignoreFirstRowSettings and the ignoreMaxResultsSettings
>>>>>>>>>>
>>>>>>>>>> Symfoware supports LIMIT, but has no equivalent for OFFSET.
>>>>>>>>>>
>>>>>>>>>> Currently, the flag setIgnoreFirstRowMaxResultsSettings toggles
>>>>>>>>>> both,
>>>>>>>>>> so either both JDBC absolute and setMaxRows are called, or 
>>>>>>>>>> both are
>>>>>>>>>> not.
>>>>>>>>>>
>>>>>>>>>> To support pagination using SQL I override 
>>>>>>>>>> printSQLSelectStatement
>>>>>>>>>> like the other platforms do, but as I only have a LIMIT in the
>>>>>>>>>> query I
>>>>>>>>>> need EclipseLink to only call the JDBC absolute method when
>>>>>>>>>> FirstRow
>>>>>>>>>> is set.
>>>>>>>>>>
>>>>>>>>>> So my implementation method looks like:
>>>>>>>>>>
>>>>>>>>>> public void printSQLSelectStatement(DatabaseCall call,
>>>>>>>>>> ExpressionSQLPrinter printer, SQLSelectStatement statement) {
>>>>>>>>>> int max = 0;
>>>>>>>>>> int firstRow = 0;
>>>>>>>>>>
>>>>>>>>>> statement.setMaximumAliasLength(getMaxFieldNameSize());
>>>>>>>>>> if (statement.getQuery() != null) {
>>>>>>>>>> max = statement.getQuery().getMaxRows();
>>>>>>>>>> firstRow = statement.getQuery().getFirstResult();
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> // only MaxRows can be worked into the statement
>>>>>>>>>> if (max > 0 && this.shouldUseRownumFiltering()) {
>>>>>>>>>> statement.setUseUniqueFieldAliases(true);
>>>>>>>>>> call.setFields(statement.printSQL(printer));
>>>>>>>>>> printer.printString(" WITH OPTION LIMIT (");
>>>>>>>>>> printer.printParameter(DatabaseCall.MAXROW_FIELD);
>>>>>>>>>> printer.printString(")");
>>>>>>>>>>
>>>>>>>>>> call.setIgnoreMaxResultsSettings(true);
>>>>>>>>>> } else {
>>>>>>>>>> // use JDBC absolute and setMaxRows methods for pagination
>>>>>>>>>> super.printSQLSelectStatement(call, printer, statement);
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Dies
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Dies Koper wrote:
>>>>>>>>>>>> Hi Tom, all
>>>>>>>>>>>>
>>>>>>>>>>>> Congrats on the 2.0 release!
>>>>>>>>>>>>
>>>>>>>>>>>> I've been continuing running the test sets on the Symfoware
>>>>>>>>>>>> platform.
>>>>>>>>>>>> May I ask you for you help with the following three issues?
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Review of patch
>>>>>>>>>>>>
>>>>>>>>>>>> I've added a patch with changes required for a number of 
>>>>>>>>>>>> tests to
>>>>>>>>>>>> run. Could you review them, and if okay, commit them?
>>>>>>>>>>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=288715
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Where is table name "SEQUENCE" defined?
>>>>>>>>>>>>
>>>>>>>>>>>> I tried renaming all occurrences of the word "SEQUENCE" in the
>>>>>>>>>>>> code
>>>>>>>>>>>> I could find but still some tests try to create a sequence 
>>>>>>>>>>>> table
>>>>>>>>>>>> with table name SEQUENCE.
>>>>>>>>>>>> It starts with the 8th test in the JPA test set:
>>>>>>>>>>>> testTransitionToDeferedFailure
>>>>>>>>>>>>
>>>>>>>>>>>> which tries a UPDATE SEQUENCE (...).
>>>>>>>>>>>>
>>>>>>>>>>>> In the log I see even more:
>>>>>>>>>>>>
>>>>>>>>>>>> [junit] [EL Config]:
>>>>>>>>>>>> ServerSession(1006920425)--Thread(Thread[main,5,main])--The 
>>>>>>>>>>>> table
>>>>>>>>>>>> generator name defined within [class
>>>>>>>>>>>> org.eclipse.persistence.testing.models.jpa.inherited.Alpine] is
>>>>>>>>>>>> being defaulted to: SEQUENCE.
>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>> [junit] [EL Fine]:
>>>>>>>>>>>> ServerSession(1006920425)--Connection(753758527)--Thread(Thread[main,5,main])--CREATE 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> TABLE SEQUENCE (SEQ_NAME VARCHAR(50) NOT NULL, SEQ_COUNT
>>>>>>>>>>>> DECIMAL(18), PRIMARY KEY (SEQ_NAME))
>>>>>>>>>>>> [junit] java.sql.SQLException: [SymfoWARE ODBC 
>>>>>>>>>>>> Driver][SymfoWARE
>>>>>>>>>>>> [...]
>>>>>>>>>>>> [junit] at
>>>>>>>>>>>> org.eclipse.persistence.testing.tests.jpa.advanced.concurrency.LifecycleJUnitTest.testSetup(LifecycleJUnitTest.java:76) 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I believe these might be from the CONCURRENCYA, B and C
>>>>>>>>>>>> tables, or
>>>>>>>>>>>> ConcurrencyA, B and C entities. Each of them has
>>>>>>>>>>>> fieldID.setIsIdentity(true) resp. @GeneratedValue specified
>>>>>>>>>>>> but the
>>>>>>>>>>>> details of the sequence generators are omitted.
>>>>>>>>>>>>
>>>>>>>>>>>> I though it would default to what the platform's
>>>>>>>>>>>> createPlatformDefaultSequence() method returns but apparently
>>>>>>>>>>>> not.
>>>>>>>>>>>>
>>>>>>>>>>>> Many tests fail because of this issue.
>>>>>>>>>>>>
>>>>>>>>>>>> 3. Table locks during DROP TABLE
>>>>>>>>>>>>
>>>>>>>>>>>> In the SymfowarePlatform wiki page you suggested that we should
>>>>>>>>>>>> start by trying to get the Core SRG running first. Most are
>>>>>>>>>>>> passing
>>>>>>>>>>>> now.
>>>>>>>>>>>>
>>>>>>>>>>>> Only a few (the ReadAllTest tests, which rely on
>>>>>>>>>>>> AutoTableGeneratorBasicTestModel.java) failed I think 
>>>>>>>>>>>> because of
>>>>>>>>>>>> the
>>>>>>>>>>>> table lock issue.
>>>>>>>>>>>>
>>>>>>>>>>>> The error message is:
>>>>>>>>>>>>
>>>>>>>>>>>> Exception Description: 24 objects were read from the database,
>>>>>>>>>>>> but
>>>>>>>>>>>> originially there were, 12.
>>>>>>>>>>>>
>>>>>>>>>>>> When I run it by itself through the test browser it passes
>>>>>>>>>>>> without
>>>>>>>>>>>> problems.
>>>>>>>>>>>>
>>>>>>>>>>>> Some messages in the finest log let me believe this is 
>>>>>>>>>>>> because a
>>>>>>>>>>>> problem
>>>>>>>>>>>> occurred while recreating the tables: they should have been
>>>>>>>>>>>> dropped and
>>>>>>>>>>>> recreated, thereby clearing those old 12 object, but because
>>>>>>>>>>>> of the
>>>>>>>>>>>> locking issue they were not.
>>>>>>>>>>>>
>>>>>>>>>>>> [junit] Internal Exception: java.sql.SQLException: [SymfoWARE
>>>>>>>>>>>> ODBC
>>>>>>>>>>>> Driver][SymfoWARE Server] JYP3913E : Table "PHONE" being used
>>>>>>>>>>>> exclusively by another user.
>>>>>>>>>>>> [junit] Error Code: -3913
>>>>>>>>>>>> [junit] Call: drop table PHONE
>>>>>>>>>>>> [junit] Query: DataModifyQuery(sql="drop table PHONE")
>>>>>>>>>>>> [junit] at
>>>>>>>>>>>> org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:324) 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [..]
>>>>>>>>>>>> [junit] at
>>>>>>>>>>>> org.eclipse.persistence.testing.models.employee.relational.EmployeeSystem.dropTableConstraints(EmployeeSystem.java:86) 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [junit] at
>>>>>>>>>>>> org.eclipse.persistence.testing.tests.schemaframework.AutoTableGeneratorEmployeeSystem.createTables(AutoTableGeneratorEmployeeSystem.java:32) 
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This table was created and accessed as follows:
>>>>>>>>>>>>
>>>>>>>>>>>> 1. drop (index) and table (outside transaction)
>>>>>>>>>>>> DROP TABLE PHONE
>>>>>>>>>>>>
>>>>>>>>>>>> 2. create table (and index) (in transaction), same connection
>>>>>>>>>>>> CREATE TABLE PHONE (EMP_ID NUMERIC(15) NOT NULL, TYPE 
>>>>>>>>>>>> VARCHAR(15)
>>>>>>>>>>>> NOT NULL, AREA_CODE VARCHAR(3), P_NUMBER VARCHAR(7), PRIMARY 
>>>>>>>>>>>> KEY
>>>>>>>>>>>> (EMP_ID, TYPE))
>>>>>>>>>>>>
>>>>>>>>>>>> 3. insert 30 rows (in single transaction), same connection
>>>>>>>>>>>> INSERT INTO PHONE (TYPE, AREA_CODE, P_NUMBER, EMP_ID) VALUES
>>>>>>>>>>>> (?, ?,
>>>>>>>>>>>> ?, ?)
>>>>>>>>>>>> [junit] bind => [Home, 613, 5551234, 105]
>>>>>>>>>>>>
>>>>>>>>>>>> 4. drop (index) and table (outside transaction), different
>>>>>>>>>>>> connection
>>>>>>>>>>>> DROP TABLE PHONE
>>>>>>>>>>>> -> SQLException saying PHONE is locked.
>>>>>>>>>>>>
>>>>>>>>>>>> I can reproduce this in a simple JDBC application. If I 
>>>>>>>>>>>> close the
>>>>>>>>>>>> connection used in steps 1-3 before step 4 it works fine.
>>>>>>>>>>>>
>>>>>>>>>>>> I can see some session swapping is done in
>>>>>>>>>>>> AutoTableGeneratorBasicTestModel#addForcedRequiredSystems() 
>>>>>>>>>>>> so I
>>>>>>>>>>>> tried adding a reset() invocation at the start of its
>>>>>>>>>>>> addForcedRequiredSystems() method but that did not help.
>>>>>>>>>>>> How can I make it close the physical connection?
>>>>>>>>>>>> (You mentioned adding logic into TestSystem?)
>>>>>>>>>>>>
>>>>>>>>>>>> The lock issue occurs much more often in the JPA test set. The
>>>>>>>>>>>> first
>>>>>>>>>>>> failing test however is because of the reserved SEQUENCE 
>>>>>>>>>>>> keyword
>>>>>>>>>>>> used as table name, so if you could help me in the right
>>>>>>>>>>>> direction
>>>>>>>>>>>> with the above three issues I can try move forward with this 
>>>>>>>>>>>> test
>>>>>>>>>>>> set again.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>> Dies
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Tom Ware wrote:
>>>>>>>>>>>>> Hi Dies,
>>>>>>>>>>>>>
>>>>>>>>>>>>> FYI: I am working on integrating your initial patch. There 
>>>>>>>>>>>>> are a
>>>>>>>>>>>>> couple of items I will not be integrating at the moment:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Specific references to SymfowarePlatform and
>>>>>>>>>>>>> isSymfowarePlatform(). These will be left until
>>>>>>>>>>>>> SymfowarePlatform
>>>>>>>>>>>>> becomes part of the full product
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Changes to default SEQUENCE table name to "\"SEQUENCE\"" 
>>>>>>>>>>>>> for
>>>>>>>>>>>>> our
>>>>>>>>>>>>> Sessions and project XML (in XMLSessionConfigProject,
>>>>>>>>>>>>> ObjectPersistenceRuntimeXMLProject and SequencingConfig).
>>>>>>>>>>>>> This is
>>>>>>>>>>>>> simply to big a backward compatibility risk since I cannot 
>>>>>>>>>>>>> test
>>>>>>>>>>>>> these changes on every database platform that EclipseLink is
>>>>>>>>>>>>> run on
>>>>>>>>>>>>> by our customers. I am looking for some way of having this
>>>>>>>>>>>>> information stored on the DatabasePlatform, but at the
>>>>>>>>>>>>> moment, it
>>>>>>>>>>>>> is not looking good. Assuming I do not find a good solution,
>>>>>>>>>>>>> Symfoware users will simply have to explicitly set these items
>>>>>>>>>>>>> when
>>>>>>>>>>>>> using our proprietary configuration code (sessions.xml and
>>>>>>>>>>>>> deployment.xml) We will also have to address anywhere in the
>>>>>>>>>>>>> tests
>>>>>>>>>>>>> we choose to run where this is an issue.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am in the process of running testing on the other changes 
>>>>>>>>>>>>> and
>>>>>>>>>>>>> I'll let you know when it gets checked in.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Tom
>>>>>>>>>>>>>
>>>>>>>>>>>>> Dies Koper wrote:
>>>>>>>>>>>>>> Hi Tom,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have added the outstanding issues with SQL keywords in
>>>>>>>>>>>>>> table/columns names, and maximum precision. The biggest open
>>>>>>>>>>>>>> issue
>>>>>>>>>>>>>> is of course the drop table restriction. I'm looking 
>>>>>>>>>>>>>> forward to
>>>>>>>>>>>>>> hearing what solution(s) you can come up with.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>>> Dies
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Tom Ware wrote:
>>>>>>>>>>>>>>> Hi Dies,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I am moving some of the main points of this discussion to 
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> wiki page so I can more easily keep track of where we are:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> http://wiki.eclipse.org/EclipseLink/Development/Incubator/Extensions/SymfowarePlatform#Open_Issues 
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I started by just adding some basics about the main issues
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> are getting in the way at the moment. I plan to add more
>>>>>>>>>>>>>>> detail
>>>>>>>>>>>>>>> next week. Please feel free to add other issues I have
>>>>>>>>>>>>>>> missed, or
>>>>>>>>>>>>>>> more detail.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As for some of the other things you were wondering about:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. Your initial patch (including sequencing fixes etc.). I
>>>>>>>>>>>>>>> hope
>>>>>>>>>>>>>>> to find some time to start integrating it next week
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2. EclipseLink bug 286907 (not related to Symfoware 
>>>>>>>>>>>>>>> platform)-
>>>>>>>>>>>>>>> this bug is in the queue and will be addressed in sequence
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> the other bugs. The fact that you have submitted a patch
>>>>>>>>>>>>>>> means it
>>>>>>>>>>>>>>> will potentially jump other bugs in the queue. At the moment
>>>>>>>>>>>>>>> I am
>>>>>>>>>>>>>>> going to focus on the Symfoware Platform issues with my free
>>>>>>>>>>>>>>> time, so I can't make any promises about when it will be
>>>>>>>>>>>>>>> included.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 3. GlassFish bug 9179 (not related to Symfoware platform) -
>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>> thing as above, but additionally: we cannot legally make
>>>>>>>>>>>>>>> use of
>>>>>>>>>>>>>>> the submitted patch until it is attached to EclipseLink 
>>>>>>>>>>>>>>> bug by
>>>>>>>>>>>>>>> someone with IP rights
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'll get started looking for some solutions next week.
>>>>>>>>>>>>>>> -Tom

Back to the top