Slowness (in fundamental mode) about yi HOT 21 CLOSED

From [email protected] on April 28, 2009 12:07:16
Also, a paste should be an atomic update.

from yi.

From [email protected] on May 06, 2009 19:05:46
I used the attached HackerMain to try to get a repeatable test. It measures the time
to insert 'x' 1000 times into a new buffer then exits.

Test 1 - Yi from darcs. Running "time yi":
real 0m9.526s
user 0m8.929s
sys 0m0.212s

Test 2 - Yi from darcs except with

hunk ./Yi/UI/Vty.hs 211

• threadDelay 100000
  (at the end of Vty.refresh)

real 0m4.311s
user 0m4.188s
sys 0m0.060s

Test 3 - Yi from darcs except with

hunk ./Yi/Core.hs 95

• withEditor $ do prepAction
• withEditor $ do --prepAction

real 0m0.879s
user 0m0.916s
sys 0m0.040s

Could calls to prepAction be avoided?

from yi.

From jeffwheeler on May 06, 2009 23:29:01
The provided HackerMain.hs file acts just like typing in tons of characters, which, admittedly, is too slow.
I wonder, though, if it is possible to get the pasteboard contents in some way other than character-by-
character?

Does anybody know what Emacs/Vim do in the terminal to access the pasteboard?

from yi.

From [email protected] on May 07, 2009 06:32:45
Probably nothing more than simply do nothing : I don't think the code for mac os and linux is too different in
emacs, and you can paste the same way to a terminal.
The difference (I guess) is that the syntax highlighting and indentation is trivial in emacs and vim, but not in yi.
That's why yi is great, that said ;-)

And when working with files a thousand lines long, typing is also quite slow and requires more cpu than in
emacs.

from yi.

From [email protected] on May 07, 2009 09:29:44
From a quick strace, emacs and vim just read in from the terminal and treat the input
as though it were typed. Emacs will read all available keyboard input using an
FIONREAD ioctl, but this is just an implementation difference.

In vim, if you :set ai, pasted code will get reindented. You can :set paste, but this
just disables autoindent. This shows that Vim processes each pasted key as if it were
typed.

If you have the rough equivalent (define-key global-map (kbd "RET")
'newline-and-indent) in .emacs, pasted code will get re-indented. Emacs has nicer
auto-indentation so doesn't mess up pastes like vim, but it's still processing each
key as though it were typed.

The attached simulates pasting LaTeX in fast-latex mode. This takes 13.7s for me,
while the same in fundamental mode takes about 9.3s. fast-latex and latex take the
same time. After commenting out prepAction, both times are under 1 second.

from yi.

From [email protected] on May 08, 2009 07:44:33
There is "theoretical" support for atomic paste in xterm; but no distributed terminal
actually implements it. I informed the xterm maintainer about this, and he might fix
it later (so in a few years it might be a common feature in terminal emulation).

Thus in practice this boils down to "yi is slow".

From there it would help to know which frontend you are using. For vty I think it
would make things snappier if the rendering was done asynchronously (in its own thread).

from yi.

From [email protected] on May 08, 2009 21:44:41
If you are using vty then can you report what version of vty you have installed? Run
the command "ghc-pkg list vty" The darcs version of vty (3.1.9) will provide higher
performance.

As for async rendering with vty: I agree entirely. I think an initial implementation
internal to the vty library should be easy.

Even with vty having performing rending on a separate thread, it may be advantageous
for yi to place the syntax processing (for instance) in a separate thread. 1. That'd
be GUI agnostic and 2. The syntax highlighting thread could be restarted when
touchedIndex is modified if the modification occurs before a highlighting update
completes. I think #2 might reduce perceived log and reduce performing highlighting
work. Though I'm not sure restarting the syntax highlighting thread on touchedIndex
modification is really required.

from yi.

From [email protected] on May 09, 2009 07:14:18
I'd like to add that in the presence of lazy evaluation, this is somewhat trickier
than it might seem. It might be that you think that the computation is done in the
"main thread", but in fact only a thunk is constructed, and all the computation
happens in the rendering thread, where the data is forced to its "linear" form.

This can be very useful actually: if we design things carefully (strictness
annotations), the syntax highlighter will run in the vty thread, without any
threading complication in Yi code.

from yi.

From [email protected] on May 09, 2009 08:52:59
In fact what you need is kind of a lazy parser : instead of making the haskellish yacc return the whole ast, yi just
needs it to return the current state of the stack automaton used to parse, and a pointer to the current position of
the stack.

from yi.

From [email protected] on May 09, 2009 09:17:50
To 11: This is already done.

from yi.

From [email protected] on May 09, 2009 12:12:48
Corey,

In my test there's no difference in runtime between vty 3.1.8 and 3.1.9, and
Graphics.Vty barely shows up in the profile: less than 10%. And it's asynchronous so
this 10% isn't even on the critical path. Yi does perform rendering in a thread
separate the editor actions - this is why adding a threadDelay to Vty.refresh speeds
up my test (though adds latency).

The bottleneck is Yi.UI.Vty.prepareAction, taking 40% inherited time and alloc. And
that 40% is much worse than it sounds because it's critical; removing it speeds up
pasting by ten times.

from yi.

From [email protected] on May 10, 2009 00:20:25
Interesting idea Jean. I will keep that in mind and make sure I do not make the
equations for queueing an update for the vty renderer too strict.

from yi.

From [email protected] on May 12, 2009 13:36:49
This small change speeds up Yi in my test from 5.5s to 3.5s. It is ugly and I don't
think it should be committed; my only point here is to demonstrate that prepareAction
is definitely performance-critical.

hunk ./Yi/UI/Vty.hs 292
tabWidth = tabSize . fst $ runBuffer win b indentSettingsB
prompt = if isMini win then miniIdentString b else ""

•    (rendered,toMarkPoint',cur) = drawText h' w
  

•    (rendered,_,cur) = drawText h' w
                           fromMarkPoint
                           point $
                           tabWidth
  

  hunk ./Yi/UI/Vty.hs 298
  ([(c,(wsty, (-1))) | c <- prompt] ++ bufData ++ [('
  ',(wsty, eofPoint))])
  -- we always add one character which can be used to
  position the cursor at the end of file

•    bufDataPlain = setSty undefined text
  

•    (_,toMarkPoint',_) = drawText h' w
  

•                            fromMarkPoint
  

•                            point $
  

•                            tabWidth
  

•                            ([(c,(wsty, (-1))) | c <- prompt] ++ bufDataPlain ++
  

  [(' ',(wsty, eofPoint))])

•                         -- we always add one character which can be used to
  

  position the cursor at the end of file
  (_, b') = runBuffer win b (setMarkPointB toM toMarkPoint')
  (modeLine0, _) = runBuffer win b $ getModeLine (commonNamePrefix e)
  modeLine = if notMini then Just modeLine0 else Nothing

For timing I'm using the LaTeX test HackerMain.hs above renamed to Main.hs, plain
"cabal configure" so no profiling, and a shellscript using the "empty" terminal
redirection tool: "empty -f -i in -o out -f ./yi && time cat out >/dev/null".

Before:

$ sh time

real 0m5.473s
user 0m0.000s
sys 0m0.000s

After:

$ sh time

real 0m3.499s
user 0m0.000s
sys 0m0.004s

from yi.

From [email protected] on May 12, 2009 15:08:01
I had another look at this and made a nicer change:

Tue May 12 16:00:42 BST 2009 [email protected]

• Make paintChars lazier
  prepareAction calls paintChars to get character positions but does not need
  syntax highlighting. Previously paintChars was too strict and forced the
  character attributes to be evaluated.

from yi.

From [email protected] on May 13, 2009 08:06:20
I applied the patch, but I'm not sure what the purpose of the "lazy" function is; can
you explain?

from yi.

From [email protected] on May 17, 2009 20:53:50
Note all editing operations are slow, for example just moving the cursor.

from yi.

From jeffwheeler on July 29, 2009 23:46:46
I think the slowness here is the same slowness affecting #86. This general slugishness, most obvious in
Pango, but present in Vty, is the biggest blocker to a release in my opinion.

from yi.

From jeffwheeler on August 07, 2009 22:32:18
This is the HackerMain.hs I've been using. With the patch I sent to the list (but
didn't commit), I can use it like this:

$ touch src/Yi/Boot.hs
$ cabal install -fhacking
$ sh HackerMain.hs -fpango

And I consistently get about 24 seconds on my machine.

from yi.

From jeffwheeler on October 17, 2009 23:30:49
The same test I was running previously now consistently crashes on my machine, after typing up to the
second or thirt line.

That is, it just hangs after some variable number of characters, and doesn't seem to post anything (that I
see) to the log or stdout/stderr.

from yi.

From [email protected] on April 27, 2011 02:48:02
This is purely subjective but it seems that Yi has been better behaved on this front. Is this still an issue for other folks?

from yi.

Performance has recently seen a very nice boost. I have no problem loading in 5000 line file and moving around/making changes. Specific Vty pasting issues are being handled as per #601. Closing.

from yi.