tdf / odftoolkit Goto Github PK

Since you're getting ready to do a new release, would it be possible to update the Jena dependency?

It's currently 3.9 from 2018, but Apache Jena is up to 3.16 now.

On this page the link "JavaDoc as bundle" leads to https://odftoolkit.org/odfdom/odftoolkit.org/api/odfdom/index.html which results in 404.

Hey there,
in your release of the odftoolkit v0.9 RC, where you wanted to skip the SimpleAPI part I found something strange. If I use the odfdom namespace with this verison, it markes an object of the OdfTable class in org.odftoolkit.odfdom.doc.table as deprecated with this explanation:

org.odftoolkit.odfdom.doc.table
Class OdfTable
Deprecated.
As of release 0.8.8, replaced by org.odftoolkit.simple.table.Table in Simple API.

( found here: https://odftoolkit.org/api/odfdom/org/odftoolkit/odfdom/doc/table/OdfTable.html )
My english might not be "the yellow from the egg", but shouldn't it be the other way round?
Like all the stuff out of the SimpleAPI namespace (org.odftoolkit.simple...) should be marked deprecated and the odfdom namespace stuff (org.odftoolkit.odfdom...) should be still considered current?
I find it a little illogical to declare the odfdom namespace in the old 0.9RC as deprecated and the simpleapi as standard, but in verison 1.0 simpleapi wont exsist anymore...

Greetings and thanks for the clarification :)

I have loaded an ODT file and trying to access the specific elements for the desired manipulation (editing and adding some data content)

But I am unable to get that

this is how I am trying to access the specific element: but I do have an issue with the OdfNamespace(), can anyone please give me any guide or a sample code for how to access a specific element to do the manipulation, if not this way or any other way please, basically I need to access the table content.

and I got the below from the Odf document.

OdfPackage pkg = OdfPackage.loadPackage(documentPath);
	OdfTextDocument odt = (OdfTextDocument) OdfDocument.loadDocument(documentPath);
	



	  XPath xpath = XPathFactory.newInstance().newXPath();
	  xpath.setNamespaceContext(new  OdfNamespace());
	  OdfFileDom dom = odt.getContentDom();
	  firstParagraph = (TextPElement) xpath.evaluate("//text:p[1]", dom, XPathConstants.NODE);

it would be really great if anybody can give me any hint or guidance, thanks.

As I understand it, the deprecation warnings in 0.9.0-RC1 for the original API should be ignored since the Simple API is going away.

Would it be possible to remove the deprecation tags so that developers aren't misled into doing a pointless migration to an API that is already dead?

Hi
Actually the only method supporting the presence of the TableTableRowGroupElement is the getNextRow and getPreviousRow of the OdfTableRow class.

Others classes like OdfTable don't manage to get the real number of rows with getRowCount or iterate among them using getRowByIndex or getCellByPosition, they all fail to considerates TableTableRowElement nodes contained by a TableTableRowGroupElement.

Best regards

Hi team members,

I would like to add ODFValidator to PHPPresentation for validating output files.

• PHPOffice/PHPPresentation#653

In my scenario, I would like that Github Actions fails if there are errors in validation.
But actually, if there errors or not the exit code equals 0.

Hello everyone,

I'm currently working with the version 0.9.0-RC1 of the odfdom-java artifact and came across the following exception when using the OdfTableColumn#setWidth(long) method (I'm trying to create a fresh spreadsheet):

Caused by: java.lang.NumberFormatException: For input string: "001,2000"
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
at java.lang.Double.parseDouble(Double.java:538)
at java.lang.Double.valueOf(Double.java:502)
at org.odftoolkit.odfdom.type.Length.parseDouble(Length.java:192)
at org.odftoolkit.odfdom.type.Length.parseLong(Length.java:164)
at org.odftoolkit.odfdom.doc.table.OdfTableColumn.getWidth(OdfTableColumn.java:146)
at org.odftoolkit.odfdom.doc.table.OdfTableColumn.setWidth(OdfTableColumn.java:175)

Walking up the stack-trace...

• If I understand itcorrectly the method also updates the relative width.
• Therefore it reads the previously set width (via OdfTableColumn#getWidth()), which internally calls the OdfElement#getOdfAttribute(OdfName) method (somehow I can't dig deeper because Eclipse says it can't debug those classes below the OdfElement). The result is an attribute column-width="001,2000in" which is cut down in 001,2000in.
• The result is parsed via Length.parseLong(String, Unit) with Unit being Unit.MILLIMETER.
• That calls Length.parseDouble(String, Unit). Here the method first removes the unit and tries to parse the value as a Double - which fails because it expects a . instead of a , as a decimal point.

Smells like - no, not teen spirit - the problem is different decimal points in different environments. As my OS is German and in Germany the decimal point is a ,, I assume that a platform dependent formatter somewhere in the code. (Un)fortunately the Double#valueOf(String) method is not platform dependent and expects a . as decimal point.

As #48 shows, validating encrypted files is not possible yet.

I have created an ODS file with LibreOffice Calc which I have protected with a password.
I opened this file as follows:

odfPackage = OdfPackage.loadPackage(document.getFile(), openPassword, null);
odfDocument = OdfDocument.loadDocument(odfPackage);

This still works. After that I try to get the MasterPage as follows:

odfDocument.getOfficeMasterStyles().getMasterPage("Default");

Unfortunately, I get this exception:

java.security.InvalidAlgorithmParameterException: Wrong IV length: must be 8 bytes long
	at java.base/com.sun.crypto.provider.CipherCore.init(CipherCore.java:525)
	at java.base/com.sun.crypto.provider.BlowfishCipher.engineInit(BlowfishCipher.java:222)
	at java.base/javax.crypto.Cipher.implInit(Cipher.java:843)
	at java.base/javax.crypto.Cipher.chooseProvider(Cipher.java:901)
	at java.base/javax.crypto.Cipher.init(Cipher.java:1433)
	at java.base/javax.crypto.Cipher.init(Cipher.java:1364)
	at org.odftoolkit.odfdom.pkg.OdfPackage.decryptData(OdfPackage.java:2052)
	at org.odftoolkit.odfdom.pkg.OdfPackage.getBytes(OdfPackage.java:1870)
	at org.odftoolkit.odfdom.pkg.OdfPackage.getInputStream(OdfPackage.java:2172)
	at org.odftoolkit.odfdom.pkg.OdfFileDom.initialize(OdfFileDom.java:219)
	at org.odftoolkit.odfdom.dom.OdfStylesDom.initialize(OdfStylesDom.java:79)
	at org.odftoolkit.odfdom.pkg.OdfFileDom.<init>(OdfFileDom.java:102)
	at org.odftoolkit.odfdom.dom.OdfStylesDom.<init>(OdfStylesDom.java:64)
	at org.odftoolkit.odfdom.dom.OdfSchemaDocument.getStylesDom(OdfSchemaDocument.java:436)
	at org.odftoolkit.odfdom.dom.OdfSchemaDocument.getOfficeMasterStyles(OdfSchemaDocument.java:536)

I am using the current code from the Master Branch.
During debugging, I became aware of the decryptData method in the OdfPackage.
There, in line 2027, the AlgorithmElement is read from the manifest.xml. It has the following content in my file:

<manifest:algorithm manifest:algorithm-name="http://www.w3.org/2001/04/xmlenc#aes256-cbc" manifest:initialisation-vector="qiWmy8seRpFOREHbrzhbag=="/>

After the Base64 decoding in line 2033, iv is then 16 bytes long.
Is it possible that in line 2048 we should use the algorithm from the manifest instead of the Blowfish? So in my case aes256-cbc?

Thanks,
Lars

hello all, I'm not entirely sure if this is the correct repo to be asking this, but the documentation/resources/websites are so very confusing!

Anyway I've been banging my head against the wall on this problem for a day now and can't seem to get it to work!

Any pointers or help would be highly appreciated!

Some context:

We have an ODF text document (this can be considered as a template), and it has a bunch of text added such as <FOO_LIST>.

What we're trying to then do is simply find all instances of the above and then programmatically inserting a bullet point list at that text location.

Simple in theory right?

Well what I have almost works, however instead of placing the bullet point list near those "place holders" they just get added to the start of the document :(

anyway here is the gist of the code (its in Kotlin):

fun BulletPoints(data: List<String>, mount: String, doc: TextDocument) {
        try {
            val search = TextNavigation("<$mount>", doc)

            while (search.hasNext()) {
                val item: TextSelection = search.nextSelection() as TextSelection
                val p = doc.getParagraphByIndex(item.index, false)

                val tempDoc = TextDocument.newTextDocument()
                tempDoc.addList().addItems(data.toTypedArray())

                doc.insertContentFromDocumentAfter(tempDoc, p, false)
            }
        } catch (e: Exception) {
            throw Exception
        }
    }

ODFDOM Getting Start Guide: http://odftoolkit.org/odfdom/index.html
Simple API Getting Start Guide: http://odftoolkit.org/simple/gettingstartguide.html
Simple API Cookbook: http://odftoolkit.org/simple/document/cookbook/index.html
Simple API Demos: http://odftoolkit.org/simple/demo/index.html
Simple API Online JavaDoc: http://odftoolkit.org/simple/document/javadoc/index.html
ODF Validator Getting Start Guide: http://odftoolkit.org/conformance/ODFValidator.html
ODF XSLT Runner Getting Start Guide: http://odftoolkit.org/xsltrunner/ODFXSLTRunner.html

Why these links broke ?
Thanks.

Now that this project requires Java >= 11, we should consider removing the dependency to xml-apis:xml-apis whose contents are already provided by the JRE for those versions of Java (I believe). It would help solving issues related to Java 9 modularity (#54) since those classes should not be exposed by two different modules.

The commits that correspond to the releases announced in the README.md are not identifiable. This makes building this releases difficult.

See https://github.com/tdf/odftoolkit/releases

Hey guys and girls,
I want to use the ODF Toolkit to work with a table (.ods file), but I don't manage to compile the project properly.
On the introduction page is written to use maven. So I downloaded it, set the enviroment path under my win 7 installation and checked for the right java sdk (its the latest java 8 - 1.8_212) set as JAVA_HOME as enviroment path. (As the Maven website describes)
I cloned the MavenCleanup Branch with this command:
git clone https://github.com/tdf/odftoolkit.git -b MavenCleanup --single-branch
into the working directory and tried
mvn clean install in it.

These Errors get shown at the end of the terminal output:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running schema2template.example.ExampleGenerationTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 88.269 sec - in schema2template.example.ExampleGenerationTest
Running schema2template.example.odf.OdfTemplateAPICoverageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in schema2template.example.odf.OdfTemplateAPICoverageTest
Running schema2template.example.odf.PuzzlePieceTest
Tests run: 3, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 0.031 sec <<< FAILURE! - in schema2template.example.odf.PuzzlePieceTest
testMSVExpressionTree(schema2template.example.odf.PuzzlePieceTest)  Time elapsed: 0.031 sec  <<< FAILURE!
java.lang.AssertionError: 
There is a difference between the expected outcome of the parsed ODF 1.0 tree.
Please compare the output:
	'target\odf10-msvtree.dump'
with the reference
	'target\classes\examples\odf\odf-schemas\OpenDocument-strict-schema-v1.0-os.rng
	at schema2template.example.odf.PuzzlePieceTest.testMSVExpressionTree(PuzzlePieceTest.java:90)

Running schema2template.TemplateAPICoverageTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec - in schema2template.TemplateAPICoverageTest

Results :

Failed tests: 
  PuzzlePieceTest.testMSVExpressionTree:90 There is a difference between the expected outcome of the parsed ODF 1.0 tree.
Please compare the output:
	'target\odf10-msvtree.dump'
with the reference
	'target\classes\examples\odf\odf-schemas\OpenDocument-strict-schema-v1.0-os.rng

Tests run: 6, Failures: 1, Errors: 0, Skipped: 2

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for The ODF Toolkit 0.9.0-SNAPSHOT:
[INFO] 
[INFO] The ODF Toolkit .................................... SUCCESS [  2.793 s]
[INFO] ODF Custom Javadoc Taglets ......................... SUCCESS [  6.218 s]
[INFO] XML Schema to Template Mapping Tool: Parent POM .... SUCCESS [  0.078 s]
[INFO] XML Schema to Template Mapping Tool: Library ....... FAILURE [01:34 min]
[INFO] XML Schema to Template Mapping Tool: Maven2 Plugin . SKIPPED
[INFO] ODFDOM ............................................. SKIPPED
[INFO] ODF XSLT-Runner .................................... SKIPPED
[INFO] ODF XSLT-Runner Ant Task ........................... SKIPPED
[INFO] ODF Validator ...................................... SKIPPED
[INFO] Simple Java API for ODF (Simple ODF) ............... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:44 min
[INFO] Finished at: 2019-05-27T15:02:27+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project schema2template: There are test failures.

It seems that the maven-surefire-plugin test doesn't work, so the complete project won't compile.
Is there another solution than skipping tests completely with
mvn install -DskipTests
or disabling something in the pom.xml ?

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.19.1</version>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
    </configuration>
</plugin>

(btw, that last tip from stackoverflow didn't work out)

Can somebody help me getting this compiled without errors?

Thanks in advance! :)

Someone who is already using Java 9 modularity might assist us in adding this concept to the ODF Toolkit JARs.

See also:
https://www.oracle.com/corporate/features/understanding-java-9-modules.html
https://www.baeldung.com/java-9-modularity

404 : /

Our existing performance (esp. of ODFDOM) should become more transparent and we should become aware of updates that influence the performance.

Therefore should basic functionality of ODFDOM (the library which is used by the ODF Validator and the Simple API depends on - partly forked) bet tested by a performance regression test.
We might want to start with something simple:

• Loading and saving a document
  other functionality as for instance table actions might follow.

The latest JDK 12 has updated the Java Micro Benchmarks to facilitate Java performace tests:
https://openjdk.java.net/jeps/230

Anybody interested in taking a look into it?

Thanks in advance,
Svante

The JavaDoc generated for 0.10.0 Beta 1 does only contain images and manifest, but no HTML, see
https://repo1.maven.org/maven2/org/odftoolkit/odfdom-java/1.0.0-BETA1/odfdom-java-1.0.0-BETA1-javadoc.jar

The 'mvn install ' on the master does not create complete JavaDoc JARs (tested for ODFDOM), either.

@mistmist You had recently taken a look at the JavaDoc (not related to JAR) would you mind taking another look in this area? ;-)

We have prepared an internal OSGi bundle for odfdom-0.8.10 and we would like to use odf-dom-1.0.0 as an OSGi bundle.
We already have a prototype running and encountered, that the crucial point is, that the java-rdfa library last released in 2012 is dependent on jena-2, which incurs a lot of historic dependencies.
I have finished the modernization of the library in the repository https://github.com/iteggmbh/java-rdfa and I'm currently discussing with Sonatype on how to gain control over the deployment to the net.rootdev groupId.
If the library needs a change in the groupId, it might be a feasible solution to change to groupId to org.odftoolkit, so the odftoolkit team might deloy a cleaned up version of java-rdfa.

Libreoffice generates an ODS file from Excel having a row with repeated-rows: 1mio and another empty one.
Reference: https://forum.lazarus.freepascal.org/index.php?topic=35451.0
This is still an issue, I found the same behavior in our ODS document.

Because of this, a
for (OdfTableRow row : table.getRowList())
iterates through a million empty rows and table.getRowCount() returns the maximum (~1 mio).

As a workaround, a repeated-rows of above, let's say 10000, should be simply ignored. There is very probably no reason to have anything repeated ten thousand times.

I did not search whether there are other occurances of getTableNumberRowsRepeatedAttribute to be regarded.
I did not research a correct way to know whether that row is "empty" in reality - even then one might add one for visuals.
I did not research the count of columns in such documents in detail, but they seem to be fixed to 1024.

It was identified by OWASP dependency-checker tool that exists vulnerabilities in odftoolkit-odftoolkit-0.9.0-RC2.
To prevent this security issues, please, upgrade to the latest version the following libraries (see image below):

On this page the link "driver" leads to http://incubator.apache.org/odftoolkit/xsltrunner/ODFXSLTRunnerTask.html which results in 404.

I have started a branch of the 0.9.0 version to support ODF 1.3.
but decided in discussions with Michael Stahl and Oliver Rau afterwards that it might be easier to have an ODF 1.3 support based solely on the upcoming ODF 1.0.0 version.
The reason for this is that 1.0.0 is the merge of a several-year-old merge form Open-XChange using ODFDOM for their backend of their new OX Documents.

In the following, I am starting a listing what needs to be fixed - this is likely being updated during the lifetime of this issue.
This documentation should be added to our SITE documentation to describe how to do an ODF version change in the future.

Changes expected so far:

1. The generator project needs to generate new ODFDOM typed DOM classes from the new ODF XML schema/grammar files. These files need to replace the ODFDOM prior classes for elements and attributes.

2. The generator tests have to be adopted to embrace ODF 1.3.

3. The ODF Validator needs to have an ODF 1.3 instead of an ODF 1.2 option.

4. The ODF templates within ODFDOM - being used when creating new documents from the scratch needs to be added to ODF 1.3.
   I did this usually by editing the XML within the ZIP using JEdit with its archive and its XML extension.

Hello everyone,

I'm currently working with the version 0.9.0-RC1 of the odfdom-java artifact (still bound to Java 1.8, so I can't use the 1.0.0-BETA1).

In OpenOffice you'll find "Split" and "Freeze" in the "Window" menu.

I went up and down the API (OdfTable, OdfTableColumn, OdfTableRow, OdfTableCell), but couldn't find any method related to that. I also created a speadsheet in OpenOffice, loaded it via the ODF Toolkit and analysed all the styles, but also couldn't find any information or clue there.

Is this a bug? A missing feature? Not in the standard of the ODS format, but in some unstandardised fields only OpenOffice knows? 🤔 Or is the problem in the front of screen? 😏

It would be helpful to have those documents updated to the upcoming API:

• https://odftoolkit.org/simple/gettingstartguide.html and
• https://odftoolkit.org/simple/document/cookbook/index.html

This issue depends on issue #87

I have simple spreadsheet template: empty template has formatted 1st row, see firstRowFormattedTemplate.zip. I tried load this template:

import org.junit.jupiter.api.Test;

import org.odftoolkit.simple.SpreadsheetDocument;
import org.odftoolkit.simple.table.Cell;
import org.odftoolkit.simple.table.Table;

import java.io.InputStream;

public class TestFromTemplate {

    @Test
    public void testWithException() throws Exception {
        SpreadsheetDocument spreadsheetDocument;
        try (InputStream is = getClass().getResourceAsStream("firstRowFormattedTemplate.ots")) {
            spreadsheetDocument = SpreadsheetDocument.loadDocument(is);
        }
        Table table1 = spreadsheetDocument.getSheetByIndex(0);
        Cell cell = table1.getCellByPosition(0, 0);


    }

}

and I catch exception:

java.lang.IllegalArgumentException: index should be nonnegative integer.
at org.odftoolkit.simple.table.Row.getCellByIndex(Row.java:370)
at org.odftoolkit.simple.table.Table.appendColumn(Table.java:1670)
at org.odftoolkit.simple.table.Table.appendColumns(Table.java:1723)
at org.odftoolkit.simple.table.Table.getCellByPosition(Table.java:2865)
at TestFromTemplate.testWithException(TestFromTemplate.java:20)

I enabled debug and saw:

I.e. lastColumnIndex is -1 and autoextend down.

Consider the following odt xml:

...
<text:p text:style-name="parent_style">
    <draw:frame draw:style-name="frame_style" draw:name="frame" text:anchor-type="paragraph" svg:x="2.589cm" svg:width="5.198cm" draw:z-index="4">
        <draw:text-box fo:min-height="0.665cm">
            <text:p text:style-name="paragraph_draw_style">Testing frame</text:p>
        </draw:text-box>
    </draw:frame>
    <text:span text:style-name="P1">Te</text:span>
    <text:span text:style-name="P2">sting</text:span>
    <text:span text:style-name="P3"> parent</text:span>
</text:p>
...

If we search for Testing, the first match will be in the sub paragraph in the frame. The next match in the parent paragraph won't be found though, as the Navigation#getNextMatchElementInTree method automatically calls nextSibling on the parent (note that it'd also be true if Testing was in a single span).
Checking in the parent if it is a paragraph shouldn't be difficult by adding a simple condition, but the tricky part would probably be avoiding re-matching the sub-paragraphs.

The other problem (due to the same assumption) appears when refreshing the selections: Assuming we currently have a selection on parent and one one frame; if we replace frame by, for example, something, the Selection#refresh static method won't refresh the parent selection, as it only considers selections that are in the same element. If we then try to replace parent, we'll get an exception.
The solution would be to also check for selections in parent elements when refreshing (which should be pretty easy), but again, the tricky part is to know if we have to refresh the parent selection or not, depending on where the child selection is regarding to the parent one.

I made a basic PR #81, but it still needs some work on the tricky parts.

This repository does not seem to have Continuous Integration set up.
As a contributor, when I open a pull request to a Java library, I expect that tests will be run automatically on it and that the maintainer will be able to merge this pull request based on those results and their review.

It would be easy to add this using GitHub Actions.

I have suggested and discussed with Michael Stahl, Oliver Rau and Thorsten Behrens to add a new branch for ODF Toolkit 1.0.0-SNAPSHOT.

I am afraid this branch is a huge jump and contains years of work on new features and fixes.

It had been triggered by the sponsoring of the German PrototypeFund in Winter 2017/2018 and I continued in the past year my work on the collaboration feature of this branch:
https://github.com/svanteschubert/odftoolkit/tree/odf-changes

In addition, this branch has many many fixes and feature enhancements from my work as a freelancer for the German Open-Xchange GmbH, which forked ODF Toolkit and use this collaboration features for the OX Documents web office and started to work on a fork before I even started there. During my working time there was never time to merge the work back...until I worked on it end of last year in my winter holidays.

The idea of the new collaboration feature in a nutshell: If multiple editors are working on copies of the same document, but would like to have a single document in the end, they need to ask the others for their changes on the document to merge them back. Unfortunately, not a single file format in the whole world is defining such changes, but only specifying the full file format for interoperability. But sending the full document for collaboration among applications is as clever as if software developers would collaborate in programming by zipping source code repositories and attaching those to emails for their co-works.

Therefore this new library is able to load an ODF Text document (ODT) and dissamble the document into an equivalent sequence of high-level user changes. Executing this sequence of user changes would create the same document, by writing it from the top to the bottom.
This list of high-level user changes is in created as JSON to be dispatched to browser offices (see the example JSON file and equal ODT file).
A browser office will receive the same changes from users as from a loaded document.

See my presentation on the LibreOffice Conference 2018 for more detailed information.

The upcoming new website should have more explicit information about this new feature and especially a high-level view from the design of this feature and its pro/cons.

Question as title
thanks

I tried to get a list of all the spreadsheet sheets on a 250k ods using OdfDocument::getTableList() and stopped the process after minutes of full cpu usage.

OdfDocument::getTableList() is using OdfSchemaDocument::getTables() is using fillTableList which is going recursively through all the XML.However, the comment above fillTableList says: "Only tables being on root level are being considered" which is not what fillTableList does.

According to the old "simple" API source and according to OdfDocument::getTableByName only the root level needs to be searched for ODS-Tables, so the comment seems correct. However, I did not look into the specification of ODS and do not know about other document formats and whether they need this method recursively.

I suggest to consider removing the recursive call from fillTableList or document it as a false-friend and create a new one for a list of ODS tables on root level.

In the meantime, it is available at https://tdf.github.io/odftoolkit/

There had been an OASIS ODF 1.3 release, but before generating the DOM classes from the XML schema (RNG grammar), it might be a good idea to regenerate the classes for ODF 1.2 again to easily spot the differences between ODF 1.2 and 1.3 (reduce the noise & be sure still everything works as it is expected).

That was supposed to be a quick task, but I had to realize that the generation was not done for a long time and many manual changes were made upon these classes.

I have created the branch odf12-codegen and addressed all the problems that I found worthy to be addressed for the next release - still, some build problems have to be fixed now manually.

There are some fundamental problems in some new hand-written classes, where some low-level XML related class is inheriting from a semantic high-level class (in theory the syntax is exchangeable and an implementation detail), which should be addressed later latest for a 1.0.

Hi,
when the Simple API will removed in the next release. The sample code must be update with ODFDOM.

https://odftoolkit.org/simple/index.html

Best regards,
Hanh

Xalan contains a nasty bug that produces incorrect XML entities in the output, leading to a corrupt document. E.g. this input

<text:span text:style-name="T19">𝜈</text:span>

Is changed to this when saving this document with odftoolkit:

<text:span text:style-name="T19">&#55349;&#57096;</text:span>

More information about the root cause can be found here:
https://issues.apache.org/jira/browse/XALANJ-2419

As it seems unlikely that there will ever be a new Xalan release including a fix for this, one option (and that is what I have been doing now) is to replace the xalan serializer dependency with a known good version, e.g.

        <dependency>
            <groupId>org.docx4j.org.apache</groupId>
            <artifactId>xalan-serializer</artifactId>
            <version>11.0.0</version>
        </dependency>

I cannot vouch for the integrity of this package but I have verified that it actually fixes the invalid encoding.

There is a lot of noise in the Java Sources of different editors doing different formatting. Therefore I would love to stabilize the syntax by a normalized.
Especially, I would like to make the differences between the sources of 0.9.0 and 1.0.0 less noisy.

Remember:
The version 0.9.0 - on the master branch - is basically our Apache (incubator) version. While the 1.0.0 - on the 1_0_0_SHAPSHOT branch - is the result of a painful manual merge of my work on a fork at Open-XChange with the Apache branch and on top the results of my work sponsored by the PrototypeFund.
Therefore, I started today testing with the Google Java Formatter:
https://github.com/google/google-java-format 
In particular, I started with the following Maven plugin:
https://github.com/Cosium/maven-git-code-format
Where I can do manual formatting on the command line via:

mvn git-code-format:format-code -Dgcf.globPattern=**/*

There are a few things I do not like very much on the result:

1. Changes within the copyright header
2. The manual formatting breaks with an out of heap error

I committed this example to a feature branch called: 
feature-formatter_variant-1_Cosium/git-code-format-maven-plugin  ;-)

Hey there,
till now I did work with the odftoolkit version 0.9.
I did compare columns of two big ODF Spreadsheet Files (.ods) with macros and a lot of other settings.
Now I wanted to update to the odftoolkit verison 1.0.0 snapshot while using the latest Java 12 (12.0.2) and get a special Error, while using the org.odftoolkit.odfdom.doc.* namespace classes.
In verison 0.9 there were no errors with the org.odftoolkit.simple.SpreadsheetDocument etc classes.

I didn't get the right clue from the IntelliJ Debugger or I don't know all capibilities yet.
The error stack trace states a heap size problem:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at org.odftoolkit.odfdom.changes.ChangesFileSaxHandler.cacheTableOperation(ChangesFileSaxHandler.java:2239)
	at org.odftoolkit.odfdom.changes.ChangesFileSaxHandler.cacheOperation(ChangesFileSaxHandler.java:2218)
	at org.odftoolkit.odfdom.changes.ChangesFileSaxHandler.endElement(ChangesFileSaxHandler.java:1722)
	at org.odftoolkit.odfdom.pkg.rdfa.MultiContentHandler.endElement(MultiContentHandler.java:90)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
	at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
	at org.odftoolkit.odfdom.pkg.OdfFileDom.initialize(OdfFileDom.java:236)
	at org.odftoolkit.odfdom.dom.OdfContentDom.initialize(OdfContentDom.java:70)
	at org.odftoolkit.odfdom.pkg.OdfFileDom.<init>(OdfFileDom.java:105)
	at org.odftoolkit.odfdom.dom.OdfContentDom.<init>(OdfContentDom.java:59)
	at org.odftoolkit.odfdom.dom.OdfSchemaDocument.getContentDom(OdfSchemaDocument.java:412)
	at org.odftoolkit.odfdom.dom.OdfSchemaDocument.getTables(OdfSchemaDocument.java:640)
	at org.odftoolkit.odfdom.doc.OdfDocument.getTableList(OdfDocument.java:737)
	at Comparison.<init>(Comparison.java:43)
	at Main.main(Main.java:6)

I did increase it till -Xms1024m -Xmx8000m in the project VM Options, but this just fills my memory more and won't help.

My steps so far:
After I cloned your 1.0.0_SNAPSHOT inside IntelliJ and build the .jars with the Maven Options 'install -DskipTests' I did only insert 'odfdom-java-1.0.0-SNAPSHOT-jar-with-dependencies.jar' as a project dependency.
Thats it. Is there something missing? It's not missing any classes, so I can't suggest anything.

I found out while stepping through the code, that inside the method 'initialize(DefaultHandler handler, OdfFileDom dom)' in 'OdfFileDom.java' it triggers the methode 'xmlReader.parse(xmlSource)' and after this invocation, my memory gets filled up till break down.
But stepping inside this .parse methode shows some decompiled xerces source files for which IntelliJ states, that they don't match the java bytecode. It won't help when I insert the Xerces_2.12.0 source files as project dependencies so I cant really look very deep into the problem.

=========================================================
As comparison I did write a very small program with the code which gets invoked till it breaks down.
To proof anything I created two new Spreadsheet Documents with only one table and a little content inside each.
With these sheets my small program doesn't have any problem to give out the tables. getTableList() is working as expected.
Sorry, I can't paste the big Spreadsheet Documents here, cause they contain sensible data.

Can someone give me any hints?
I hope I didn't produce a unique error there...

I'll stay with my 0.9 builds, till that issue is solved. I can't build working jar files from the odftoolkit master branch here at the moment 🙄

Thanks in advance
eHx

EDIT:
With xerces-j in the dependencies before odftoolkit it goes deeper into the java libs:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.base/java.util.LinkedList.addAll(LinkedList.java:428)
	at java.base/java.util.LinkedList.addAll(LinkedList.java:391)
	at java.base/java.util.LinkedList.<init>(LinkedList.java:123)
	at org.odftoolkit.odfdom.changes.ChangesFileSaxHandler.cacheTableOperation(ChangesFileSaxHandler.java:2239)
	at org.odftoolkit.odfdom.changes.ChangesFileSaxHandler.cacheOperation(ChangesFileSaxHandler.java:2218)
	at org.odftoolkit.odfdom.changes.ChangesFileSaxHandler.endElement(ChangesFileSaxHandler.java:1722)
	at org.odftoolkit.odfdom.pkg.rdfa.MultiContentHandler.endElement(MultiContentHandler.java:90)
	at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source)
	at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanEndElement(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
	at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
	at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
	at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
	at org.odftoolkit.odfdom.pkg.OdfFileDom.initialize(OdfFileDom.java:236)
	at org.odftoolkit.odfdom.dom.OdfContentDom.initialize(OdfContentDom.java:70)
	at org.odftoolkit.odfdom.pkg.OdfFileDom.<init>(OdfFileDom.java:105)
	at org.odftoolkit.odfdom.dom.OdfContentDom.<init>(OdfContentDom.java:59)
	at org.odftoolkit.odfdom.dom.OdfSchemaDocument.getContentDom(OdfSchemaDocument.java:412)
	at org.odftoolkit.odfdom.dom.OdfSchemaDocument.getTables(OdfSchemaDocument.java:640)
	at org.odftoolkit.odfdom.doc.OdfDocument.getTableList(OdfDocument.java:737)
	at example.simple.Comparing.showavailableTables(Main.java:52)
	at example.simple.Main.main(Main.java:20)

Originally posted by @0x000eHx in #13 (comment)

Hi,

Are the maling lists links still correct?

I mean the archives links are, but for new messages?

The Readme stil references them.

Thank you.

P.S. Any chance activating a gitter room https://gitter.im for the project? It integrates quite simply with GitHub.

Hi,

Any chance getting a new release to Maven?

Thank you.

Generated ODS file; may or may not be in the correct format. Calc opens it just fine.

export.ods: Fatal: A qualified name was required, but 'manifest' was given! java.lang.IllegalArgumentException: A qualified name was required, but 'manifest' was given! at org.odftoolkit.odfdom.pkg.OdfNamespace.splitQName(OdfNamespace.java:142) at org.odftoolkit.odfdom.pkg.OdfNamespace.getPrefixPart(OdfNamespace.java:154) at org.odftoolkit.odfdom.pkg.OdfName.newName(OdfName.java:82) at org.odftoolkit.odfdom.pkg.OdfFileDom.createElementNS(OdfFileDom.java:298) at org.odftoolkit.odfdom.pkg.OdfFileSaxHandler.startElement(OdfFileSaxHandler.java:82) at org.odftoolkit.odfdom.pkg.rdfa.MultiContentHandler.startElement(MultiContentHandler.java:76) at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source) at org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source) at org.apache.xerces.impl.XMLNSDocumentScannerImpl$NSContentDispatcher.scanRootElementHook(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.odftoolkit.odfdom.pkg.OdfFileDom.initialize(OdfFileDom.java:236) at org.odftoolkit.odfdom.pkg.OdfFileDom.initialize(OdfFileDom.java:212) at org.odftoolkit.odfdom.pkg.manifest.OdfManifestDom.initialize(OdfManifestDom.java:73) at org.odftoolkit.odfdom.pkg.OdfFileDom.(OdfFileDom.java:132) at org.odftoolkit.odfdom.pkg.manifest.OdfManifestDom.(OdfManifestDom.java:64) at org.odftoolkit.odfdom.pkg.OdfFileDom.newFileDom(OdfFileDom.java:194) at org.odftoolkit.odfdom.pkg.OdfPackage.parseManifest(OdfPackage.java:1313) at org.odftoolkit.odfdom.pkg.OdfPackage.readZip(OdfPackage.java:549) at org.odftoolkit.odfdom.pkg.OdfPackage.initializeZip(OdfPackage.java:519) at org.odftoolkit.odfdom.pkg.OdfPackage.(OdfPackage.java:328) at org.odftoolkit.odfdom.pkg.OdfPackage.loadPackage(OdfPackage.java:439) at org.odftoolkit.odfvalidator.ODFStreamValidator.getPackage(ODFStreamValidator.java:52) at org.odftoolkit.odfvalidator.ODFRootPackageValidator.getPackage(ODFRootPackageValidator.java:59) at org.odftoolkit.odfvalidator.ODFPackageValidator._validate(ODFPackageValidator.java:99) at org.odftoolkit.odfvalidator.ODFPackageValidator.validate(ODFPackageValidator.java:84) at org.odftoolkit.odfvalidator.ODFValidator.validateStream(ODFValidator.java:211) at org.apache.jsp.jsp.validate_jsp._jspService(validate_jsp.java:232) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:477) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:712) at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:580) at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:516) at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:955) at org.apache.jsp.jsp.index_jsp._jspService(index_jsp.java:128) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:477) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:688) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:834)

Attached file looks like medical data but it's all junk data from a dev box.
export.zip

Hello, can you check if this issue is resolved/fixed? They say it was fixed but we use this API in a project and the memory is leaking.

https://issues.apache.org/jira/browse/ODFTOOLKIT-300?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%3D16369140#comment-16369140

The changes of whitespaces (trailing spaces, tabs vs. spaces) are handled by many code editing application different and cause unintended noise for reviewers of commits.
Even I got problems myself when using multiple editors such as Netbeans, JEdit, UltraEdit and VSCode.

I suggest to add some automated Maven/GitHook tooling to our system, which normalizes all whitespaces. In other words, to avoid noise for reviewers every commit of an author is being space-normalized before it reaches GitHub.

It would be very desireable to have this tooling before we do our next release to keep the delta to follow up releases as low as possible.

The ODF Validator is has a dependency

org.iso_relax.verifier.jaxp.validation
isorelax-jaxp-bridge
1.0

Which has a broken pom.xml on Maven Central:
http://central.maven.org/maven2/org/iso_relax/verifier/jaxp/validation/isorelax-jaxp-bridge/1.0/isorelax-jaxp-bridge-1.0.pom
^^There is no element, but only

The previous work around was to load a correct version from a different Maven repro:

apache.smx.repo
http://svn.apache.org/repos/asf/servicemix/m2-repo/
Apache ServiceMix Repo

false


true

Unfortunately, Travis does not do this.
Thanks for Oliver Rau, who worked with mit on this problem for over an hour to figure out the root problem... :-)

The simple work around was to use an upgraded version on Maven central with a valid pom:

I have compared the sources from above with the original sources here:
https://github.com/kohsuke/isorelax-jaxp-bridge

The only change is that one class was made public to provide reflexion. Works for us! :-)

Please, add ability to write validator errors and to System.err. And using System.exit with nonzero error code, if validator errors exists (for checking validator erros in batch mode).

Please clarify the Java version for this project.

The Getting Started (and other seachable articles) still mention Java 6, but the pom has Java 8.

Thank you.

Links to download .jar-files in ODF TOOLKIT RELEASE NOTES are broken.

The links for documentation are broken:

• The Home Page for the ODF Toolkit
• ODFDOM Getting Start Guide
• ODF Validator Getting Start Guide
• ODF XSLT Runner Getting Start Guide

There have to be a high-level API that abstracts from the implementation details of the XML DOM as Simple API & org.odftoolkit.odfdom.doc API but now in addition has to be as well compatible with the new change (collaboration) approach
https://tdf.github.io/odftoolkit/odfdom/operations/operations.html
(every operation has to be mappable to one or more API calls - best in a generic way (of naming & finding these methods))

This new operation/change approach allows every ODT document to be transformed into an equivalent list of user operations/changes as if the document has been just created from top to bottom by the user. Such a change list in JSON can be currently created by any ODFDOM user of the latest repository (or BETA) calling the JAR with a JDK >=9 (tested with JDK11)

java -jar odfdom-java-1.0.0-BETA1-jar-with-dependencies.jar <USER'S ODT>

Or can be found in the following as an [example]:(https://github.com/tdf/odftoolkit/blob/master/docs/docs/presentations/character-styles.odt) and JSON.

The reason and advantage of switching from a final state (zipped ODT document state) to a more fine granular user-change concept is to be able to answer the most important question of collaboration & to be able to do a merge & synchronize: "What have you changed?".

On top of this higher level SDOM API will be in addition some Search API to query the content of one or more document(s), e.g.

• return all visible text (to be translated)
• return a reference to all tables
• what changes had been made in a period of time (e.g. during my vacations)

Excel seems to generate ODS files in a way which confuses this library (0.9.0-RC1).
LibreOffice can import this ODS file just fine.
made-with-excel.ods.zip

Opening this file with odftoolkit and calling OdfTableRow.getCellCount() on its rows seems to be running in an infinite loop.