peek_analysis_map_inline does not work for active space about latex3 HOT 7 CLOSED

I'll try in two weeks time. Right now work is especially hectic.

from latex3.

Correct, I should have clarified that in the commit message. Since I was touching this code anyways I figured it was simplest to do both.

from latex3.

@blefloch Any chance you could look at this?

from latex3.

The current code is the following.

\ExplSyntaxOn
  \cs_new_protected:Npn \__tl_peek_analysis_exp:N #1
    {
      \cs_set_nopar:Npe \l__tl_peek_code_tl
        {
          \tex_let:D \exp_not:N #1 \scan_stop:
          \exp_not:o \l__tl_peek_code_tl
          {
            \exp_not:n { \__kernel_exp_not:w \exp_after:wN }
              { \exp_not:N \exp_not:N \exp_not:N #1 }
          }
          \exp_after:wN \__tl_peek_analysis_exp_aux:Nw
          \token_to_str:N #1 \prg_do_nothing: \s__tl
        }
      \l__tl_peek_code_tl
    }


  \cs_new:Npe \__tl_peek_analysis_exp_aux:Nw #1#2 \s__tl
    {
      \exp_not:N \if:w \scan_stop: #2 \scan_stop:
        { \exp_not:N \int_value:w ‘#1 } \token_to_str:N D
      \exp_not:N \else:
        { -1 } 0
      \exp_not:N \fi:
    }

The reason seem obvious:
if \token_to_str:N #1 consists of a single character ("space"),
we want the #2 to expand to empty so first branch is taken,
which it indeed is taken (because #2 expands to empty), but the problem is #1 becomes
\prg_do_nothing: and \int_value:w `\prg_do_nothing: gives an error.

A silly patch I come up with is

\ExplSyntaxOn
  \cs_new:Npe \__tl_peek_analysis_exp_aux:Nw #1#2 \s__tl
    {
      \exp_not:N \if:w \scan_stop: #2 \scan_stop:
        { \exp_not:N \int_value:w \exp_not:N \exp_after:wN `#1 \c_space_tl } \token_to_str:N D
      \exp_not:N \else:
        { -1 } 0
      \exp_not:N \fi:
    }

so that it becomes \int_value:w \exp_after:wN `#1 ⟨space⟩. Because the number syntax is `⟨character⟩ ⟨optional space⟩,
if #1 is anything else it will be eaten up happily, while if it is \prg_do_nothing: then the \exp_after:wN will get rid of it.

Problem? Possible performance degradation. (There's the extra \exp_after:wN and the extra ⟨space⟩ to be executed every time)

from latex3.

Thanks a lot for pointing me to exactly the right place! This sped up the investigation. I found a tighter fix that avoids \prg_do_nothing: altogether but the basic idea is the one you gave: use that the backtick syntax has an optional trailing space. Now committed to develop.

from latex3.

Speaking of which there's a little performance optimization possible here.

        \exp_not:o \l__tl_peek_code_tl

could just be

\l__tl_peek_code_tl

because at that point in time \l__tl_peek_code_tl only contains a \protected token \__tl_analysis_map_⟨number⟩:nnN.

Although of course if the token to be peeked is equal to \__tl_analysis_map_⟨number⟩:nnN itself, it will break. A simple fix I can find is to edit the \scan_stop: above to \l__tl_peek_code_tl which makes performance a bit worse (my benchmark says that the degradation is less than that of the performance again obtained by the removal of \exp_not:o above), but this token is obviously internal to the implementation so up to you whether to implement it.

from latex3.

@blefloch Thanks (just to clarify, the optimization and the break are separate, you can keep the first one be scan_stop and remove the exp_not:o in the second one)

from latex3.