Git Product home page Git Product logo

libflexport's Introduction

FINDOLOGIC export toolkit

Github Actions Latest Stable Version Code Climate Codecov

Table of Contents

  1. Synopsis
  2. Export recommendation
  3. Limitations
  4. Basic usage
    1. Setup
    2. XML Export
    3. CSV Export
  5. Examples
  6. Compatibility
  7. Contributors

Synopsis

This project provides an export library for XML and CSV generation according to the FINDOLOGIC export patterns.

Export recommendation

Using the XML export is recommended by FINDOLOGIC. The XML is easier to read and has some advantages over the CSV export like:

  • No encoding issues as the encoding attribute is provided in the XML response <?xml version="1.0" encoding="UTF-8"?>.
  • Validation is more reliable.
  • Simple escaping of content using the <![CDATA[...]]>-tag.
  • Standardized structure.
  • Dynamically extract the products from the database via start and count parameter in the url.
  • No limited file size for XML because of pagination.
  • Using multiple groups per product.

The key advantage for CSV is that it is possible to use way more groups than for XML. On the other hand:

  • Groups only regulate visibility - it's not possible to show different values per group.
  • The format is prone to encoding issues if non-UTF-8 data is fed into it.
  • Total export size is limited by file size, while XML pagination theoretically allows exports of arbitrary size.

Limitations

Currently, only input text encoded in UTF-8 is supported. To use this library with other types of encoding, one of the following is necessary:

  • Convert all text to UTF-8 prior to passing it to libflexport.
  • Use the XML exporter and modify the library to change the XML header to contain the required encoding.
    • FINDOLOGIC is capable of handling most encodings, but only with XML.

Basic usage

Setup

  1. Include as composer dependency using composer require findologic/libflexport
  2. Load ./vendor/autoload.php into the project

XML export

require_once './vendor/autoload.php';

use FINDOLOGIC\Export\Exporter;

$exporter = Exporter::create(ExporterType::XML);

$item = $exporter->createItem('123');

$item->addName('Test');
$item->addUrl('http://example.org/test.html');
$item->addPrice(13.37);
// Alternative long form:
// $name = new Name();
// $name->setValue('Test');
// $item->setName($name);
// $url = new Url();
// $url->setValue('http://example.org/test.html');
// $item->setUrl($url);
// $price = new Price();
// $price->setValue(13.37);
// $item->setPrice($price);

$xmlOutput = $exporter->serializeItems([$item], 0, 1, 1);

CSV export

require_once './vendor/autoload.php';

use FINDOLOGIC\Export\Exporter;

$exporter = Exporter::create(ExporterType::CSV);

$item = $exporter->createItem('123');

$item->addPrice(13.37);
$item->addName('Test');
$item->addUrl('http://example.org/test.html');
// Alternative long form:
// $name = new Name();
// $name->setValue('Test');
// $item->setName($name);
// $url = new Url();
// $url->setValue('http://example.org/test.html');
// $item->setUrl($url);
// $price = new Price();
// $price->setValue(13.37);
// $item->setPrice($price);

// Date is mandatory for CSV.
$item->addDateAdded(new \DateTime());

$csvOutput = $exporter->serializeItems([$item], 0, 1, 1);

Examples

For more specific examples, please have a look at the examples directory.

Compatibility

The status of the major versions of libflexport is outlined below. Version numbers generally follow semantic versioning principles.

Version Branch PHP support Receives bug fixes Receives enhancements End of life
3.X develop >=8.1 ✔️ ✔️ Not in the foreseeable future
2.X 2.x >=7.1 ✔️ Not in the foreseeable future
1.X 1.x 5.6 - 7.3 ✔️ TBD
0.X 5.6 - 7.0 2017-11-24

All versions will most likely remain available for as long as the infrastructure to do so exists.

Contributors

See contribution guide.

libflexport's People

Contributors

dombra avatar howard avatar mmachatschek avatar mohamadassani avatar thekeymaster avatar tobiasgraml11 avatar zaifastafa avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libflexport's Issues

Support properties (extra columns) in CSV export

  • Format as tab-separated values.
  • Cause exception when trying to assign a value containing tabs.
    • Maybe it's better to just replace tabs in values with spaces. Less transparent, but also less error prone. I'm open to suggestions.

Use import statement for top level classes

Is your feature request related to a problem? Please describe.

Currently the imports of top level classes are mixed.

Describe the solution you'd like

Use import statements for all top level classes in each file so it's not mixed anymore.
Also add configuration for the linter so it's also part of the build process.

Describe alternatives you've considered

No alternatives

Attribute values containing multidimensional data will cause an mb_srtlen exception

Describe the bug

Passing a multidimensional array to the Attribute instance, will cause an mb_strlen exception.

To Reproduce

// Simply create a new Attribute instance and pass it multidimensional array data:
new \FINDOLOGIC\Export\Data\Attribute(
    'attribute_with_very_long_value',
    [
        'ayy' => [
            'interesting' => 'nice'
        ]
    ]
);

Expected behavior

A clear exception indicating I have passed wrong data there, which isn't valid.

Actual behavior

An mb_strlen exception happens, which isn't really clear about what happened:

mb_strlen() expects parameter 1 to be string, array given
 /home/dom/github/findologic/libflexport/src/FINDOLOGIC/Export/Helpers/DataHelper.php:101
 /home/dom/github/findologic/libflexport/src/FINDOLOGIC/Export/Data/Attribute.php:37
 /home/dom/github/findologic/libflexport/src/FINDOLOGIC/Export/Data/Attribute.php:46
 /home/dom/github/findologic/libflexport/src/FINDOLOGIC/Export/Data/Attribute.php:28

Screenshots

image

Support the new visibility element.

Is your feature request related to a problem? Please describe.

In the dependency xml-export-schema >= 1.6.0, a new optional element for visibility is added.

Describe the solution you'd like

  • XML:
    The visibility element looks like:

    <visibilities>
        <visible>true</visible>
        <visible usergroup="foo">false</visible>
    </visibilities>

    Check this PR for more insides.

  • CSV:
    To copy XML, a new optional column visible must be added to CSV as well.

  • For both:
    visible must contain a lowercased true (or 1 as a replacement) or a lowercased false (or 0 as a replacement).

visibility must be supported, and would be great to include test-coverage.

Bonus, DateAdded, SaleFrequency and Sort are needed for the XML.

If I want to generate the XML with libflexport I need to set at least a value for following Elements:

$item->setBonus(new Bonus("0"));
$item->setSalesFrequency(new SalesFrequency("0"));
$item->setDateAdded(new DateAdded("0"));
$item->setSort(new Sort("0"));

If I don't, I will get a PHP Fatal Error:

Call to a member function getDomSubtree() on null in pathTo/findologic/export/src/FINDOLOGIC/Export/XML/XMLItem.php on line 35

The lines are 35, 36, 37, and 38.

The FINDOLOGIC XML schema mentions that those values are optional, but If I don't set them in the library, I will get an error.

If you need more information or you can not reproduce this case, please contact me.

Regards,
TheKeymaster

Allow empty values according to FINDOLOGIC's XML schema

There are only two parameters required in the XML schema according to the documentation.
But the library requires also things like ordernumber, name etc.
Since it is a library for the FINDOLOGIC Export, it should also ignore it when there is no ordernumber or name given.
The following parameters are required from the library, but not from the XML schema:

  • Name
  • Summary
  • Description
  • Url
  • Ordernumber

Everything else should be fine. That's what I've found so far.

Clean up

  • Fix formatting in README.md.
  • Create a unit test for XML that breaks when more items are supplied than specified in $count.
  • Comment that the $count parameter for CSV, albeit required, is ignored.

Support prices with a value of 0 for XML and CSV export

  • For customers that want export prices like 0.0. This is important for the DokuWiki export and for customers that export not only products, but also content pages.

Cause: \FINDOLOGIC\Export\Helpers\DataHelper::checkForEmptyValue

if (empty($value)) {
    throw new EmptyValueNotAllowedException();
}

Fix: Soon

start, count and total should not be required method arguments when using CSV

Describe the bug

\FINDOLOGIC\Export\Exporter::serializeItems and \FINDOLOGIC\Export\Exporter::serializeItemsToFile require $start, $count and $total method arguments even though they are not required (as stated in the docblock).

Since they are required some value has to be filled in order to make the IDE happy. This should not be necessary.

To Reproduce

Steps to reproduce the behavior:

  1. Library version 'v1.3.0'
  2. PHP version '7.1'
  3. Use a proper IDE such as PhpStorm.
  4. Call \FINDOLOGIC\Export\Exporter::serializeItems or \FINDOLOGIC\Export\Exporter::serializeItemsToFile without passing the ignored paramters $start, $count and $total.

Expected behavior

  • The IDE does not complain.

Actual behavior

  • The IDE complains about required parameters missing.

Support price validation

libflexport allows any type of string (except it is empty) to be put in the XML. This could result in an validation issue, because the XML Scheme only allows a decimal.
The export allows following:

cvc-datatype-valid.1.2.1: 'blubbergurken' is not a valid value for 'decimal'.

A potential fix would be to have an own price validation that checks for is_float().

Better information in exception messages

For projects where the call stack of the exception is not printed out, it is hard to determine which part of the code is responsible for problems like Empty values not allowed!.

For this reason it should be possible to see more detailed infomation in the exception message e.g.

Empty values are not allowed in "Attribute", "Key/Value" etc.

Maybe it would be also helpful to print out an item id if possible.

Support special prices for CSV export

  • instead
    • Format with decimal dot.
  • maxPrice
    • Format with decimal dot.
  • taxRate
    • In percent points.
  • Ensure type safety in the setter to avoid illegal values being exported.
  • Not supported by XML, so when trying to generate XML from an item with those set, an exception should occur.

Remove non-printable characters

In some cases the provided string values can contain non-printable characters. Those do not necessarily break the resulting XML but render them invalid.

It would be nice if libflexport could take care of those, since this issue isn't 100% implementation specific.

Adding attributes without values

Describe the bug

Adding attributes to an item with emty values should be prohibited.

To Reproduce

  1. Create item with mandatory elements
  2. Create attribute without values
  3. Assign the attribute to the item
  4. Get the XML or CSV of the page

Expected behavior

The export breaks with an exception when adding the attribute without values.

Actual behavior

An invalid XML is generated which doesn't comply to our XSD-Schema

'salesFrequencies': Missing child element(s). Expected is ( salesFrequency )

Describe the bug

I got an error message that my XML is not valid because it's missing a child element called 'salesFrequency' within the 'salesFrequencies' element. The XSD requires this element and its corresponding value. The 'salesFrequenciesType' also requires a non-negative integer and an optional 'usergroup' attribute.
However, the XML documentation states that a tag is required but a value is not.

Expected behavior

The xml is valid without an salesFrequency value

Actual behavior

I got an error: "DOMDocument::schemaValidate(): Element 'salesFrequencies': Missing child element(s). Expected is ( salesFrequency )"

P.S. I got the same error for allOrdernumbers: "DOMDocument::schemaValidate(): Element 'allOrdernumbers': Missing child element(s). Expected is ( ordernumbers )."

Adding invalid urls to image elements

At the moment it's possible to add URLs for image elements which are not valid. The XSD-schema restricts the content of the <image /> elements to be urls.

This code should case a exception:

$item->addImage(new Image('.store.com/blubbagurke.jpg'));

Customizable encoding for XML export

It should be possible to set the encoding of the XML document as this feature is available by the FINDOLOGIC schema and documented in the official documentation.

Add support for variants

Is your feature request related to a problem? Please describe.

  • Findologic is starting to support variant search as a first class citizen.
  • This requires a specifically structured export, which libflexport doesn't support yet.
  • Currently, variants are only supported with CSV export, but from libflexport's perspective, this should work transparently for all supported output formats.

Describe the solution you'd like

  • libflexport allows creating CSV- and XML exports with- or without variants in a similarly simple and intuitive manner.
  • A good solution might require interface changes, so this could be a candidate for the next major release.

Describe alternatives you've considered

  • Manual export implementation, as it's done currently for variants, is still an option until a larger number of people start using them.

Document methods

All methods that are used to create the export should be documented (docblock) to add hints when using e.g. a IDE

before FINDOLOGIC\Export\Data\Attribute::setValues()

public function setValues($values)
{
    $this->values = [];

    foreach ($values as $value) {
        $this->addValue($value);
    }
}

after FINDOLOGIC\Export\Data\Attribute::setValues()

/**
 * Set values allocated to this attribute.
 *
 * @param array $values The collection of values allocated to this attribute.
 */
public function setValues($values)
{
    $this->values = [];

    foreach ($values as $value) {
        $this->addValue($value);
    }
}

Add PHP 7 type declarations

e.g.

class DataHelper
{
    /**
     * Checks if the provided input complies to a FINDOLOGIC valid url.
     * URL needs to have a http[s] schema.
     * See https://docs.findologic.com/doku.php?id=export_patterns:xml#urls
     *
     * @param string $url The input to check.
     * @throws InvalidUrlException If the input is no url.
     * @return string Returns the url if valid.
     */
    public static function validateUrl(string $url)
    {
        if (!filter_var($url, FILTER_VALIDATE_URL) || !preg_match('/http[s]?:\/\/.*/', $url)) {
            throw new InvalidUrlException();
        }

        return $url;
    }
}

Add PHP 7 return types

Void types should not be forgotten either http://php.net/manual/en/migration71.new-features.php#migration71.new-features.void-functions

e.g.

class Property
{
    /**
     * Add a value to the property element.
     *
     * @SuppressWarnings(PHPMD.StaticAccess)
     * @param string $value The value to add to the property element.
     * @param string|null $usergroup The usergroup of the property value.
     */
    public function addValue($value, $usergroup = null)
    {
        if (array_key_exists($usergroup, $this->getAllValues())) {
            throw new DuplicateValueForUsergroupException($this->getKey(), $usergroup);
        }

        $this->values[$usergroup] = DataHelper::checkForEmptyValue($value);
    }
}

Missing group parameter

Is your feature request related to a problem? Please describe.

A clear and concise description of what you want to happen.

  • The best solution in my opinion is that there is a method to set the parameter group

A clear and concise description of any alternative solutions or features you've considered.

  • Alternative there should be a comment in the docs

Support groups for CSV export

  • Re-use Usergroup.
  • Implement UserGroup::getCsvFragment to output group names as a comma-separated list.
    • Throw an exception in case a usergroup contains a comma. This should not impact XML.

More specific validation of URLs

It is possible to pass a image URL link like https://www.xyz.de/media/image/3a/83/55/88780_image1[1].jpg to the library which doesn't throw an error. The XSD Schema doesn't allow this value. This cases should be handled by the library.

Missing `use` statement?

An EmptyValueNotAllowedException may be thrown in \FINDOLOGIC\Export\Helpers\UsergroupAwareSimpleValue::validate.

Although it shouldn't in some cases this produces a class not found exception. Maybe you are required to use namespaces although the exception class is in the same?

Collect ideas for libflexport 3.0

libflexport 3.0 - where are we going?

A new version of PHP with exciting features was released recently, and we've now had quite some time to collect experience about working with libflexport from internal- and customer perspective. It's time to apply this experience and leverage new possibilities to provide the best libflexport so far.

What's (mostly) certain

  • libflexport 3.0 will have PHP 7.4 as minimum requirement.
  • Maintainability level A must be restored.
  • Input validation must stay at the same level, if not increase.
  • All aspects of the library will be strictly typed.

Contributed ideas

  • More consistent handling of groups and usergroups.
  • Support for variants.
  • Possibly tied to XML 2 and CSV 3 export formats.

Impact on 1.x and 2.x

  • Support for both 1.x and 2.x will for now remain at the same level as now.
  • Once 3.x is sufficiently stable, and the adoption of PHP 7.4 has reached a TBD level, the support for 2.x will be reduced to bug fixes.

What we need before getting started

  • What works well with the current structure and usage of libflexport?
  • What can be improved?
    • Which things just outright suck?
    • Where are inconsistencies?
    • What is not intuitive or too complicated?
  • How can we reduce friction when creating an export from scratch?

Call to action

Please comment below if you have any input for the next major version of libflexport. Even the wildest ideas are welcome - they might not be part of the final thing, but at least inform its design. Your input is critical, so the next version is top instead of flop.

Extend string value validity check

At the moment the sanity check only covers empty strings (I guess). If one provides an object of a class instead of a string, no exception will be thrown but the resulting value will still be empty.

Simple function for basic data

At the moment I need three lines in order to set a name without a usergroup:

$name = new Name();
$name->setValue($string);
$item->setName($name);

There should exist an easy way to handover the data to the exporter:

$item->setNameValueWithoutUsergroup($string);

Don't mind my function name there. I know it is bad.

Why should this be implemented?

This reduces the overhead immense when you want to really fill in much attributes, variations, properties etc. might even be >80 lines of code in one function, which is not needed. The function may still do the same internally, but is way easier to call.

Why not refactoring yourself?

Refactoring myself would be also very impracticable, because either a create a function for each XML element, or I create a function where I also need to add the class Name as a parameter in order to do such calls, which results in a complete fiesta.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.