Git Product home page Git Product logo

php-xbase's Introduction

PHP XBase

Build Status Test Coverage Latest Stable Version Total Downloads License

A simple library for dealing with dbf databases like dBase and FoxPro. It's a port of PHPXbase class written by Erwin Kooi, updated to a PSR-2 compliant code and tweaked for performance and to solve some issues the original code had.

Installation

You can install it through Composer:

$ composer require hisamu/php-xbase

Sample usage

More samples in tests folder.

Reading data

use XBase\TableReader;

$table = new TableReader('test.dbf');

while ($record = $table->nextRecord()) {
    echo $record->get('my_column');
    //or
    echo $record->my_column;
}

If the data in DB is not in UTF-8 you can specify a charset to convert the data from:

use XBase\TableReader;

$table = new TableReader(
    'test.dbf',
    [
        'encoding' => 'cp1251'
    ]
);

It is also possible to read Memos from dedicated files. Just make sure that .fpt file with the same name as main database exists.

Performance

You can pass an array of the columns that you need to the constructor, then if your table has columns that you don't use they will not be loaded. This way the parser can run a lot faster.

use XBase\TableReader;

$table = new TableReader(
    'test.dbf', 
    [
        'columns' => ['my_column', 'another_column']
    ]
);

while ($record = $table->nextRecord()) {
    echo $record->my_column;
    echo $record->another_column;
}

If you know the column type already, you can also call the type-specific function for that field, which increases the speed too.

while ($record = $table->nextRecord()) {
    echo $record->get('my_column');
    echo $record->get('another_column');
}

Editing Data

To open a table for editing, you have to use a TableEditor object, as on this example:

use XBase\TableEditor;

$table = new TableEditor('test.dbf');

for ($i = 0; $i < 10; $i++) {
    $record = $table->nextRecord();
    
    $record->set('field', 'string');
    //or
    $record->field = 'string';

    $table->writeRecord();
}

$table
    ->save()
    ->close();

Add new record

use XBase\TableEditor;

$table = new TableEditor(
    'file.dbf',
    [
        'editMode' => TableEditor::EDIT_MODE_CLONE, //default
    ]
);
$record = $table->appendRecord();
$record->set('name', 'test name');
$record->set('age', 20);

$table
    ->writeRecord()
    ->save()
    ->close();

Delete record

use XBase\TableEditor;

$table = new TableEditor('file.dbf');

while ($record = $table->nextRecord()) {
    if ($record->get('delete_this_row')) {
        $table->deleteRecord(); //mark record deleted
    }    
}

$table
    ->pack() //remove deleted rows
    ->save() //save changes
    ->close();

Creating table

To create a table file you need to use the TableCreator object.

use XBase\Enum\FieldType;
use XBase\Enum\TableType;
use XBase\Header\Column;
use XBase\Header\HeaderFactory;
use XBase\TableCreator;
use XBase\TableEditor;

// you can specify any other database version from TableType
$header = HeaderFactory::create(TableType::DBASE_III_PLUS_MEMO);
$filepath = '/path/to/new/file.dbf';

$tableCreator = new TableCreator($filepath, $header);
$tableCreator
    ->addColumn(new Column([
        'name'   => 'name',
        'type'   => FieldType::CHAR,
        'length' => 20,
    ]))
    ->addColumn(new Column([
        'name'   => 'birthday',
        'type'   => FieldType::DATE,
    ]))
    ->addColumn(new Column([
        'name'   => 'is_man',
        'type'   => FieldType::LOGICAL,
    ]))
    ->addColumn(new Column([
        'name'   => 'bio',
        'type'   => FieldType::MEMO,
    ]))
    ->addColumn(new Column([
        'name'         => 'money',
        'type'         => FieldType::NUMERIC,
        'length'       => 20,
        'decimalCount' => 4,
    ]))
    ->addColumn(new Column([
        'name'   => 'image',
        'type'   => FieldType::MEMO,
    ]))
    ->save(); //creates file

$table = new TableEditor($filepath);
//... add records 

Troubleshooting

I'm not an expert on dBase and I don't know all the specifics of the field types and versions, so the lib may not be able to handle some situations. If you find an error, please open an issue and send me a sample table that I can reproduce your problem, and I'll try to help.

Useful links

Xbase File Format Description

File Structure for dBASE 7

DBF AND DBT/FPT FILE STRUCTURE

php-xbase's People

Contributors

aaronhuisinga avatar angelhappyboy avatar cdyweb avatar cuchac avatar developer88 avatar ebta avatar gam6itko avatar gerryd avatar honginho avatar jlindsey3 avatar josev814 avatar kovinet avatar louisroy avatar luads avatar martonmiklos avatar naderskhan avatar retnek avatar risingphoenix avatar s-chizhik avatar spdionis avatar xmorave2 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

php-xbase's Issues

moveTo($offset) gives the wrong row with some DBF files

Hi there,

So, I've been trying to figure out what's happening with little luck.
I take a DBF file with over 30k rows and loop over it 1000 rows at a time, and the further it gets the more overlap I get with the rows between two loops.

Between the 6th and 7th loop it's 2 rows of overlap, and between the 30th and 31st it's around 700, out of the 1000 returned per loop. It takes the script I have an offset of 49000 to read all the 33000 rows in the DBF file.

For example, when reading just one row, offsets 40000 and 40001 return the same row.

When I open the file in LibreOffice and with dbview I get the same, correct, row count between them.

I've been trying to debug the script with no luck. Unfortunately I can't give you the original DBF file to debug because of sensitive information in it. However, I am willing to try and find the issue myself if you have some pointers on what the issue could be.

To make matters weirder, if open the DBF file with LibreOffice and just save it, then xbase handles offsets correctly. This makes me assume that there is odd formatting or encoding in the original file which gets squashed once LibreOffice saves it.

Any pointers?

Missing end-of-file marker

According to the Data File Header Structure for the dBASE Version 7 Table File:

The end of the file is marked by a single byte, with the end-of-file marker, an OEM code page character value of 26 (0x1A).

I do not notice this marker on any files I generate with the program, and a search for 0x1a, 1a or 26 in this library does not reveal anything that actually writes this marker.

Strangely, most programs I use to open these files don't seem to mind, but it seems like it's better to be safe than sorry.

Exception error

Testing your write code. Get the Exception:
PHP Fatal error: Call to undefined method XBase\Table::openWrite()

Dangerous number_format with implicit parameters

https://github.com/hisamu/php-xbase/blob/26ceae6c7ca67d67132b5c1108e5e2906343631d/src/XBase/Record.php#L449

Function number_format without defined parameters dec_point and thousands_sep does not work properly for DBF number, because these parameters are taken from locales. Fof example thousand separator can ba a space, so it generate format such this: "1 000". And this value is not valid.

A correct call must be: number_format($value, $columnObj->getDecimalCount(), '.', '')

Failed to read Memo field at version 131

i got a problem while read the memo type field the result always 1 char.

$data = ($table->nextRecord()); $data->getMemo('remark1')

result is "2" the actual result should be "(03/08) NOTA DEBET DANAMON Rp
SENTRA MULTI S - 200 PCS KARDUS SB"

thanks.

Install Error

Composer gives me this error when trying to install "Failed to decode response: zlib_decode(): data error"

Date format

Hi; I want to thank you for this amazing code. It's helping me a LOT.

However, I'm having problems with fields that are of DATE data type. Somehow instead of getting them in php as "04/31/2017" when I use it (and the actual foxpro DBF table has it saved in this format) I receive "Wednesday, April 31 2017 00:00:01".

How do I set up the date format when receiving the value from the DBF?

Thank you

Floating point values read error

I'm developing a project that reads an ancient FoxPRO DB, converts it and pushes data into a MariaDB database. One of the former versions in 'dev-master' had possibility to decode floating point numbers correctly. Now something changed, and after 'composer update' instead of '8.0' I'm getting '6,3380589665485E-10'. Here is the file: SP_REF.ZIP. The column in question is 'STVREF'.

Unfortunately I cannot rollback to a previous stable version of php-xbase because of issue #21.

EDIT: here is the commit that introduces the error: 59ce03b

can not read dbf

i have dbf file with 223 columns and 902 records.
i use this library to read in php, but it can not read the records of that dbf file and only can read the columns of that dbf file. why?

Some columns cannot be read

I have some .dbf files in work which i need to fill the database but one file with memo file while opened with Libre office calc have 12 columns but luads/php-xbase see only 3 of them. I am sending problematic files as an attachment and also here is the link for my testing laravel project for easier simulation of the problem many thanks for any help.

https://github.com/Fabricio872/foxproTest

foxproTest.zip

How i can get a row?

Hello! In my task i need to get rows how assoc array and analize them! I dont know name column i must to get information from row.
This is realy to get row how assoc array key value and analize this array?

Can't be read single row tables

Tables with single row can't be read with $table->nextRecord()
I think there's an error (table.php line 138) I would change it for:

        if (($this->recordPos + 1) **>** $this->recordCount) {
            return false;
        }

Memo lookup overly matchy

When looking up the memo file, the library will replace any instance of "DBF" or "dbf" in the table file path, not just in the extension. This means that if the path has those strings elsewhere in the path, the memo file won't be located, as the overzealously modified path likely won't exist.

Problems with Date fields

Hello, I am having problems to save the date with this format d/m/Y, when I echo the field the result is bool(false) @kingofnull would you help.
kind regards

Currency format

You can easily add currency format, this works for us:

const DBFFIELD_TYPE_CURRENCY = 'Y';    // Currency

public function getCurrency($columnName)
{
    $s = $this->choppedData[$columnName];

    $s = unpack('q', $s);

    if ($s) {
        return $s[1] / 10000;
    }

    return 0;
}

Class not found error when try to use XBase lib in PHP

I am trying to use https://github.com/hisamu/php-xbase lib in my project.
I have copied XBase folder from repository to the my root and created index.php in my root.

When I try this code:

<?php

use XBase\Table;

$table = new Table(dirname(__FILE__) . 'data/test.dbf');

while ($record = $table->nextRecord()) {
    echo $record->my_column;
}

?>

I received this error:

Fatal error: Class 'XBase\Table' not found in ...

What is wrong?

Record->forceGetString trims data

I don't think manipulation of the content of the file should happen within this script. Trimming should probably happen in userland code.

For my case, I removed the trim() call in forceGetString() as it was causing problems with expected offsets. I'm not sure if this could be a widespread issue, but my regular tests, with other files, didn't break when I removed trim().

What do you think?

Add version tag for composer.

Currently php-xbase does not have a version tag, so you cannot lock a specific version for php-xbase in your application composer.json file.

Could you please add a version tag?

Beware don't trust type in column as INT.

This is not real issue for this libary. This library is very good and help my life but some of dbf files that I work with it have some problem.

When they parsed and tell it as Number but in real data has decimal so Right now in my migration script. I must make all of them as FLOAT to MySQL and make second migration to make decision for write correct value to new application.

I read in source code and author use byte reading method so I think this is best solution right now. (I don't know another method too.)

For who that use this library. You must recheck your data before use.

Thank again for making this lib. :)

Pagination

i have dbf file with 10k rows how to do pagination?
or limit results to 10,20 / page

Can't read Memo type from Foxpro

Hi,

I'm iterating through a FoxPro database and one of the columns is of type 'memo'.
It seems xbase has a problem with this data type : printing the value shows a question mark inside a black diamond.

Do you have a solution or is it a known problem that will be fixed one day ?

Thank you,
Corentin.

image

PHP7 support

Hi, thank you very much for the library, it's going to be very helpful in a project I'm working on now.

I had a quick look at the code and it looks like it will be compatible with php7, which I am using now (that's why I'm using this lib and not the extension).

I will test it out in practice to see if everything works well in practice later.

I suggest changing the php version requirement in composer.json to explicitly note that it supports php7 (after I try it out of course). Leaving the issue here to not forget to notify you if everything runs fine :).

Dates seem to have bad encoding

It appears that when using a date field, T or D, the encoding makes the data useless. For example, I have a datetime of 2010-03-17 15:56:52 when pulling data from the ODBC connector. When I use this library to import the data into MySQL I noticed all the data fields were 0000-00-00 respectively. So I dumped the data to see what the row looks like before I send it to MySQL and I am getting date results like: év%'b"�. This appears to only when used from this library and only on date fields. Fetching the data with the ODBC shows the data as expected and so does using it from a DBF viewer. I guess I will have to just use the ODBC driver for my imports :( Was really hoping to use this as it's faster. Hopefully this will help you find the issue. I tried to see if I could convert it back to the original value my changing the encoding myself but no luck there.

Upload 0.1.0 stable release to packagist

Right now it is not possible to install "hisamu/php-xbase 0.1.0" via composer because the only available version is "dev-master". Any buggy updates to it will cause dependent software to break, and it is not possible to specify a stable version to protect software from possible bugs and api changes.

Missing columns

I import a dbf-file with 65 columns. After the import I miss the last 9 columns. The file is a foxpro-file (with version-number 245 from readHeader-method).

My workaround is a modification of the __construct-method by adding a custom argument named $missingColumns. After that I add this value to the $fieldCount-variable in the readHeader-method:

$fieldCount = floor(($this->headerLength - ($this->foxpro ? 296 : 33)) / 32); if ($this->missingColumns) $fieldCount += $this->missingColumns;

That works for me. Maybe the file file has too much columns?

Integer values bad read

Hello,

I have some DBF files from Foxpro, and i faced issue with bad reading integer files.

In Record.php method getIndex should read the integer, but return bad value for me...
According to this documentation, the "I" type of column should be Integer, is it what you mean by "Index"?
For me the "I" type should be read like this:

public function getIndex($columnName) {
    $su = unpack("i", $s);
    return $su[1];
}

I could open a pull request, but I want to know someone else's opinion and also I want to know the original intence of this method.

0 numeric value, return false

Please check
public function getNum($columnName)

Record.php, line 224
if (!$s) { return false; }

When dbf column having zero value, $s will have "0"
then this code will return false

Is it supposed to return 0?

is it 'case-sensitive'

I'm tryint to open a dbf file called DATA.DBF, but haven't been successful so far.

I wanted to know if the file has to be named in lowercase (data.dbf) or if that doesn't matter, in order to look for some error in my code or in my server configuration.

Thank you

Can’t insert dates in DBF

Hi all,

Please guide me into how to insert dates in the DBF files. Every time I add a record, the DBF fields that are of the “Date” value always go either null, 0000/00/00 or with the initial date of unix epoch 31/12/1969.

Exactly which is the correct way to insert a date in the date field of the DBF file??

I hope you can help me with this, thanks

Support for indexing?

Is there currently any support for indexing (CDX files)? Writing to tables works fine, other than it doesn't update its corresponding .cdx file, which is used by our legacy application.

Are there any plans to support this soon?

License?

I do not see a license for this library. Thus, it's not exactly clear what the terms of use are of this library at a glance.

I checked on PHPClasses, and it seems like the original library's license is labeled "Freely Distributable". It's kind of vague, and it doesn't account for the work you and others have done on the library since then. Github docs have this to say about source without a license.

Would it be possible to select a license for this project? It seems like an MIT license would be in the spirit of that original "Freely Distributable" language, but I'll leave that to your discretion.

Not all field types in header are read with foxpro file

In Table.php method readHeader(), the fieldcount is calculated by this formula: $fieldCount = floor(($this->headerLength - ($this->foxpro ? 296 : 33)) / 32);
However, I have foxpro files that do not contain the extra 'Database Container' block at the end of the header so because calculated number of fields is too low and some of the fields at the end of the list are not read.

In fact, one does not need to calculate the fieldcount at all since the field list is always terminated by a 0x0D byte. You do need to peek 1 byte ahead for every field you read to see if it's 0x0D, if not then it's part of the fieldname and you can do an fseek of -1.

how do I delete a record?

Hi all,

I'm working with a DBF and sometimes I need to 'delete' a record from it. How do I do it?

Cannot properly open FoxPro db.

I have several FoxPro dbs from some accounting software. I need to open it and grab the info.

After my fix i managed to open the file and read some data, but lots of columns (especially with type B, N, F, D) do not received properly - either some corrupted data or just NULL.

Some info about db file from the Table.php:

foxpro: 1
version: 48
encrypted:
mdxFlag:
languageCode:
modifyDate: 1443484800
recordCount: 20426
recordByteLength: 991
inTransaction:
headerLength: 2504
backlist:
encoding: i guess it is 1251, at least data from columns type C is properly encoded.

Is php-xbase able to work with this version of FoxPro db?

Wrong number recorded

Hi! I have the following problem.
This is my sql database table:

| id | record_id | from | num | qty | qty2 | to |
| 150 | 34 | 111 | 200006 | 555.000 | 5550.000 | 222 |

When I try to write this data in dbf the value of the column qty2 is inserted as 5.000. The problem is present when the value is more than 999.000 (has 4 numbers before the decimal). If the value is 9000.000 it is written as 9.000. The script is taking only the first digit of the number. The header (with type) of the dbf colum qty2 is qty,N,14,3

I'm using this code to write the dbf:

$table = new WritableTable($files_path.'/file.dbf', null, 'CP1251');
$table->openWrite();
$lines = Records::all();
foreach ($lines as $line) {
$record = $table->appendRecord();
$record->from = $line->from;
$record->num = $line->num;
$record->qty = $line->qty;
$record->qty2 = $line->qty2;
$record->to = $line->to;
$table->writeRecord();
}
$table->close();

Any advice on how to solve this will be much appreciated. Thank you!

Date in far future

Cannot read date in far future - e.g. 1/1/2055. Problem with strtotime and timestamp, maybe?

can't create table

I can't figure out how to create a table.
1st attempt:

$dbf = new WritableTable($dbfFile);
$dbf->create($dbfFile, $fields);

Table::__construct calls open which assumes the table exists and throws

2nd attempt:

$dbf = WritableTable::create($dbfFile, $fields);

PHP fails
Non-static method XBase\WritableTable::create() should not be called statically, assuming $this from incompatible context

if I set create as a static function, but it won't work fine either, because it uses $this

I can modify your lib, but I want to try it as it is first.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.