Git Product home page Git Product logo

jxbase's Introduction

JxBase

Build Status codecov

JxBase is a fork from jdbf, a Java utility to read/write DBF files

Convert a DBF file to SQLite from command line

Call, from command line, the class com.github.jferard.jxbase.tool.DatabaseLoader. Here's an example:

...$ java -cp "$HOME/.m2/repository/com/github/jferard/jxbase/0.0.1-SNAPSHOT/jxbase-0.0.1-SNAPSHOT.jar" com.github.jferard.jxbase.tool.DatabaseLoader
Usage: java -cp 'path/to/jxbase/jar:path/to/jdbc/driver/jar' com.github.jferard.jxbase.tool.DatabaseLoader [option] [source] [connect_string]

   -h, --help          Print this message
   -c driver_class     Load the driver using Class.forName
   -d                  Drop tables if they exist
   -s N                Chunk size (default is one chunk of the size of the file)
                       Use this to avoid an out of memory for big files
   source              A directory or a single dbf file
   connection_string   A connection string to the database
...$java -cp "$HOME/.m2/repository/org/xerial/sqlite-jdbc/3.32.3.2/sqlite-jdbc-3.32.3.2.jar:$HOME/.m2/repository/com/github/jferard/jxbase/0.0.1-SNAPSHOT/jxbase-0.0.1-SNAPSHOT.jar" com.github.jferard.jxbase.tool.DatabaseLoader -d -s 10000 $HOME/prog/java/jxbase/src/test/resources/data1 "jdbc:sqlite:./test.sqlite"
oct. 17, 2020 10:52:55 PM com.github.jferard.jxbase.tool.DatabaseLoader buildAndFillTables
INFOS: Build tables
oct. 17, 2020 10:52:55 PM com.github.jferard.jxbase.tool.DatabaseLoader buildAndFillTable
INFOS:  > /home/jferard/prog/java/jxbase/src/test/resources/data1/gds_im.dbf
oct. 17, 2020 10:52:55 PM com.github.jferard.jxbase.tool.DatabaseLoader buildAndFillTable
INFOS:  > /home/jferard/prog/java/jxbase/src/test/resources/data1/tir_im.dbf

User Guide

Read a DBF file

TODO

See the integration test example

Read a DBF file with MEMO fields

TODO

See the integration test example

Write a DBF file

TODO

See the integration test example

Write a DBF file with MEMO fields

TODO

See the integration test example

Design of an xBase database

The database is a simple set of files.

References

See:

The DBF file

The main file. Contains the header and the records.

+------- Header ---------+
|       Metadata         |   <- XBaseMetaDataReader/Writer    
| Field Descriptor Array |   <- XBaseFieldDescriptorArrayReader/Writer
|        Optional        |   <- XBaseOptionalReader/Writer
+------------------------+
|        Records         |   <- XBaseRecordReader/Writer
+------------------------+

DBT file (memo: optional)

This file contains block of data.

+------------------------+  \
|         Header         |   |
+------------------------+   } <- XBaseMemoReader/Writer
|        Records         |   |
+------------------------+  /

NDX file (index: optional)

TODO

Design of JxBase

JxBase aims to build DBF format reader and writer by layers. Each dialect derives an existing format and adds fields, formats, files.

The covariance parameter problem

Whereas a GenericRecordReader has a GenericDialect parameter, a FoxProRecordReader needs a FoxProDialect parameter. A covariant parameter (the parameters type varies with the class type) is a real problem because it violates the Liskov substitution principle: the subclass FoxProRecordReader won't work with a GenericDialect parameter (see https://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)#Covariant_method_parameter_type). Usually, one solves this problem with generics:

    class GenericRecordReader<D extends GenericDialect> {
        ...
    }
    
    class FoxProRecordReader extends GenericRecordReader<FoxProDialect> {
        ...
    }

Currently, JxBase doesn't use generics for covariance because this would lead to a really complex situation. The fallaback used to achieve the covariance is a simple cast: we accept a GenericDialect and try to cast the instance into a FoxProDialect

jxbase's People

Contributors

dependabot[bot] avatar jferard avatar

Stargazers

avatar

Watchers

avatar avatar

jxbase's Issues

Can't delete *.dbt file after creation

If you try to delete *.dbf and *.dbt file, after line dbfWriter.close() in WriterWithMemoIT , you will find that you can not delete *.dbt file, "because it used by another process". As i can see it happens because MemoWriter is not closed during dbfWriter.close()

Add BytesUtils.equals

Before:

if (DB4Utils.MEMO_FIELD_RESERVED_BYTES[0]` != recordHeaderBytes[0] ||
            DB4Utils.MEMO_FIELD_RESERVED_BYTES[1] != recordHeaderBytes[1] ||
            DB4Utils.MEMO_FIELD_RESERVED_BYTES[2] != recordHeaderBytes[2] ||
            DB4Utils.MEMO_FIELD_RESERVED_BYTES[3] != recordHeaderBytes[3]) {

After:

if (BytesUtils.equals(MEMO_FIELD_RESERVED_BYTES, 0, recordHeaderBytes, 0, 4)) {

Replace makeLEInt by extractLEInt

Before:

    final int memoRecordLength =
            BytesUtils.makeLEInt(recordHeaderBytes[4], recordHeaderBytes[5], recordHeaderBytes[6],
                    recordHeaderBytes[7]);

After:

    final int memoRecordLength = BytesUtils.extractLEInt(recordHeaderBytes, 4, 4);

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.