xclud / dart_bidi Goto Github PK

Hi
A bug has appeared on line 95 of src/paragraph.dart (_removeBidiMarkers method):
RangeError (index): Invalid value: Valid value range is empty

To Reproduce:

print(bidi.logicalToVisual('\u200E+'));

DavBfr/dart_pdf#1524

Hey, @xclud
context: There are two letter forms in Arabic that render exactly the same, the basic form and the isolated form.

Notices how the following Arabic letter LAM has the same glyph for both basic and isolated form.

currently, the algorithm maps to the isolated form which might be the right thing to do but unfortunately Arabic Google fonts seem to have a problem with this, this issue can be fixed by always mapping to the basic form when an isolated form is required.

even dart seems to work with the basic form in such cases

 final letter = 'ل';
 print(letter.codeUnits); // prints [1604]
 print(bidi.logicalToVisual(letter)); // prints [65245]

I'm willing the make a PR, but do you think we should introduce this change as an optional change or just hard-code it?

P.s I'll be changing the mappings inside of shapping_resolver._charForms

Hi, any plans to convert this code to null-safe?

I am facing an issue with the Dart Bidi package related to the incorrect handling and rendering of specific Kurdish characters. The issue arises when using the Bidi algorithm to manage and display text that contains these characters:

U+06B5 (Kurdish LAM WITH SMALL V: ڵ)
U+06CE (Kurdish FARSI YEH WITH THREE DOTS BELOW: ێ)
U+06D5 (Kurdish AE: ە)

Context

I have used several fonts that support Kurdish, including NotoSansArabic and NotoSansKurdish. These fonts work perfectly in the FlutterFlow UI and in other environments like MS Word.

Initially, I suspected that the problem was related to the dart_pdf package, and I spent considerable time investigating and debugging the issue. After cloning the dart_pdf repository and reviewing the code extensively, I found that the bug was not in dart_pdf itself, but in the Dart Bidi package, specifically in how the bidi algorithm processes these Kurdish characters.

Additionally, I discovered a related issue on the dart_pdf GitHub repository from 2013 (Issue #1408: Bug in bidi text rendering), which describes a similar problem with the Bidi algorithm not handling certain character sets correctly. This bug still affects current projects, and it is essential for this issue to be addressed.

What I Have Tried
Cloned and investigated dart_pdf: I initially thought the issue was due to dart_pdf. After spending time cloning the repository and reviewing the Bidi implementation within dart_pdf, I realized that the root of the problem was in how the Dart Bidi package processes these Kurdish characters. Because i found that dart_pdf Applies THE BIDIRECTIONAL ALGORITHM using (https://pub.dev/packages/bidi)

Modified various Bidi settings: I experimented with different configurations within the dart_bidi package, including altering character shaping logic and adjusting text handling strategies. However, the incorrect shaping of Kurdish characters persisted.

Despite these efforts, I have been unable to find a solution to this issue, and it appears to be a bug within the dart_bidi package itself.

Similar Issues:
I noticed that similar issues have been reported in other frameworks, which may help better understand the problem because they are exactly the same. (e.g., Stimulsoft and Aspose), so it seems like a broader Unicode rendering issue.

Just for making the problem easier to fix:
In kurdish ڵ is just treated as ل in arabic
In kurdish ێ is just treated as ئ in arabic
In kurdish ە is just treated as ة in arabic

Hello @xclud in your latest refractor you made paragraph.paragraphSeparator private, and I'm using it in some package, can you please make it public again?

first of all, thank you for this great package.
I've noticed that when a letter of the sentence has a HAMZA an error occurs.
I think the problem is letters with Hamza should be reshaped not decomposed into a base letter and a diacritic.

All of these throw an out of range error

  bidi.logicalToVisual('أ');
  bidi.logicalToVisual('إ');
  bidi.logicalToVisual('ؤ');
  bidi.logicalToVisual('ئ');

stacktrace

RangeError (index): Invalid value: Only valid value is 0: 1
dart:core                               List.[]
package:bidi/src/paragraph.dart 752:24  Paragraph.internalCompose
package:bidi/src/paragraph.dart 113:5   Paragraph.normalizeText
package:bidi/src/paragraph.dart 42:5    Paragraph.text=
package:bidi/src/paragraph.dart 13:10   new Paragraph
package:bidi/src/bidi.dart 73:13        splitStringToParagraphs
package:bidi/src/bidi.dart 9:16         logicalToVisual
test/arabic_test.dart 40:8              main

these don't throw an error but the result is incorrect

  bidi.logicalToVisual('خطأ');
  bidi.logicalToVisual('كؤ');
  bidi.logicalToVisual('مئ');