Questions about oniguruma HOT 19 CLOSED

1. Outside character classes, characters that must be escaped to be used literally are: ^$.?*+()[|
   Yes.
2. In character classes, characters that must be escaped to be used literally are: ^-]
   plus [
3. But in some other implementation (Atom) that use Oniguruma, [ can be used unescaped in classes but ] can't.
   No.
   I don't know what syntax Atom uses, but I suppose that is ONIG_SYNTAX_DEFAULT.
   [ in char-class means start of nested char-class. You must escape [ as normal character.
   You must escape ] in char-class unless it appears at the top of char-class.

static void warn_func(const char* s)
{
  fprintf(stderr, "WARN: %s\n", s);
}

extern int main(int argc, char* argv[])
{
  onig_set_warn_func(warn_func);

  exec(ONIG_ENCODING_UTF8, ONIG_ENCODING_UTF8, ONIG_OPTION_NONE,
       "[]a]", "]");

  return 0;
}

(* omit exec() code)

result:
WARN: character class has ']' without escape: /[]a]/
match at 0  (UTF-8)
0: (0-1)

from oniguruma.

✨ Thanks very much for the clarification! ✨

https://github.com/kkos/oniguruma/wiki/Characters-That-Must-Be-Escaped-to-Be-Used-Literally Did I miss anything?

And could you keep this open just in case I have further questions? :))

from oniguruma.

Thank you.
The content is correct.

from oniguruma.

Isn't \w equivalent to [0-9A-Z_a-z]? I thought it was, but I found it can match much much more than that.

from oniguruma.

\w is equivalent to [0-9A-Z_a-z] if you are using ASCII encoding.
\w matches many code points in Unicode(UTF-8, UTF-16, UTF-32) encoding.
Unicode word code points data is defined at CR_Word[] in src/unicode_property_data.c

from oniguruma.

So in UTF-8 encoding, \w can match the characters that are mapped to the code points defined in CR_Word[], right?

from oniguruma.

Yes. And 654 is the number of code ranges.

from oniguruma.

Thanks for your replies. They really helped me a lot!

So similarly, in UTF-8 encoding, \d can match stuff specified in CR_Digit[], right? And can I assume [0-9_A-Za-z] is "faster" than \w in UTF-8?

Do you mind if I help edit the doc of RE? It would just be some small refinements to make it easier to understand.

from oniguruma.

I do not know the difference of the speed�.
Because single byte code range and multi byte code range are separated in compiled regexp code, it may be such not different.

I have add you as collaborator.
Please edit files in doc.

from oniguruma.

And please edit develop branch not master branch.

from oniguruma.

👌 😉

from oniguruma.

About \G, does it mean "where the current match attempt begins", i.e., either \A or "where the last match left off"?

from oniguruma.

onig_search() has search-range argument (start, range) and string argument (str, end).
Search-range is matching start position range.

\G mean start position of search-range.
\A mean start position of the string.
In most cases, search-range and string are same value, then \G == \A.

from oniguruma.

Tried digging around the definition of onig_search, I could understand only a part of it.

I was trying to expand on/clarify some definitions in doc/RE, one of them is of \G:

\G matching start position

\G is useful in a regexp applied to the same string more than once. To define it from the user's standpoint. The first time the regexp applied, it matches the beginning of string; later it matches where the last match ends, right? So I wanna change the definition to something like

beginning of the current search attempt

Does it look correct to you?

from oniguruma.

Yes. You are right.

from oniguruma.

I was wondering, apart from that [:...:] is only available in character class, what really is different between Unicode properties and the POSIX notation, why do we need them both? I mean, Unicode properties are so much more powerful, why do we still need the POSIX notation?

from oniguruma.

Thank you for improved document.

POSIX bracket is poor than Unicode property.
It is not necessary anymore now.
But it was included in GNU regex library.
My first goal was to make GNU regex compatible library, and thereafter I was introduced character property function etc.. from Perl.

from oniguruma.

👌 No problem, and I can get to know more about how Oniguruma processes regexps along the way. 😌 So are you planning to deprecate POSIX brackets?

New question: in doc/RE,

In the back reference by the multiplex definition name,

What is a multiplex definition name?

Mind if I email you with the questions instead?

from oniguruma.

I will remove POSIX bracket from version 7.0 if it is removed in Perl6.
But I don't know it is removed or not.

You can assign one name to the more than two groups.

from oniguruma.