Postgres Pro
Downloads
Home   >   mailing lists

Thread: bizgres - bizgres: Enhancements and Fixes 1.

bizgres - bizgres: Enhancements and Fixes 1.

From
nwakefield@pgfoundry.org (User Nwakefield)
Date:
Log Message:
-----------
Enhancements and Fixes
1. NIOFileReader has been reworked behind the scenes to solve a severe problem with record delimiters existing within a
field.From now on the character stream is read field by field, with an unsynchronized buffer existing between the file
channeland the reading class. Performance should be comparable, with complete support for multi character delimiters. 
2. FastSimpleDateFormat, had a problem with its internal cache that resulted in fields with the same value matching
eventhough they were not in the same position 
3. Parameter write back from the SQL JOb executor has been migrated from MPP. This allow for SQL jobs to writeback  to
parameterlist. See the IncrementalFetch.xml for an example of its usage. 
4. Added RunNIOFileReaderTestjobs.java, this is a simple JUnit test suite that tests the reader.

Modified Files:
--------------
    bizgres/KETL/src/com/kni/etl/dbutils:
        Upsert.java (r1.3 -> r1.4)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/Upsert.java.diff?r1=1.3&r2=1.4)
        DatabaseColumnDefinition.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/DatabaseColumnDefinition.java.diff?r1=1.2&r2=1.3)
        ColumnDefinition.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/dbutils/ColumnDefinition.java.diff?r1=1.2&r2=1.3)
    bizgres/KETL/src/com/kni/etl:
        BaseDataItem.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/BaseDataItem.java.diff?r1=1.2&r2=1.3)
        DataItem.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/DataItem.java.diff?r1=1.2&r2=1.3)
        SourceFieldDefinition.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SourceFieldDefinition.java.diff?r1=1.1&r2=1.2)
        EngineConstants.java (r1.6 -> r1.7)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/EngineConstants.java.diff?r1=1.6&r2=1.7)
        SQLJob.java (r1.4 -> r1.5)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SQLJob.java.diff?r1=1.4&r2=1.5)
        SQLJobExecutor.java (r1.3 -> r1.4)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/SQLJobExecutor.java.diff?r1=1.3&r2=1.4)
        FastInputChannel.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/FastInputChannel.java.diff?r1=1.2&r2=1.3)
    bizgres/KETL/src/com/kni/etl/ketl/writer:
        SubComponentParallelBatchWriter.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/SubComponentParallelBatchWriter.java.diff?r1=1.1&r2=1.2)
        JDBCWriter.java (r1.6 -> r1.7)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/JDBCWriter.java.diff?r1=1.6&r2=1.7)
        PGBulkWriter.java (r1.4 -> r1.5)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/PGBulkWriter.java.diff?r1=1.4&r2=1.5)
        ETLWriter.java (r1.2 -> r1.3)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/ETLWriter.java.diff?r1=1.2&r2=1.3)
        SubComponentBatchRunnerThread.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/writer/SubComponentBatchRunnerThread.java.diff?r1=1.1&r2=1.2)
    bizgres/KETL/src/com/kni/etl/stringtools:
        FastSimpleDateFormat.java (r1.1 -> r1.2)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/stringtools/FastSimpleDateFormat.java.diff?r1=1.1&r2=1.2)
    bizgres/KETL/src/com/kni/etl/ketl:
        ETLStep.java (r1.6 -> r1.7)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/ETLStep.java.diff?r1=1.6&r2=1.7)
    bizgres/KETL/src/com/kni/etl/ketl/reader:
        NIOFileReader.java (r1.5 -> r1.6)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/reader/NIOFileReader.java.diff?r1=1.5&r2=1.6)

Added Files:
-----------
    bizgres/KETL/src/com/kni/etl:
        FieldLevelFastInputChannel.java (r1.1)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/FieldLevelFastInputChannel.java?rev=1.1&content-type=text/x-cvsweb-markup)
    bizgres/KETL/src/com/kni/etl/ketl/test:
        RunNIOFileReaderTestJobs.java (r1.1)

(http://cvs.pgfoundry.org/cgi-bin/cvsweb.cgi/bizgres/bizgres/KETL/src/com/kni/etl/ketl/test/RunNIOFileReaderTestJobs.java?rev=1.1&content-type=text/x-cvsweb-markup)