The plan is to change the Text data structure to be line-ending-agnostic, and have code that calls it deal with it pre-separated arrays of strings. The editor state should store the line separator somewhere (#1) and that's what you get when you read text from the editor, and what's used to split when you put text in. The default mode should again be a loose one where everything that looks like a newline is split on, and the output uses \n characters.

One frustration I had while developing a custom theme was that brackets and parentheses are not spans with classes, so are not themeable as separate entities. They are lumped together with commas as "punctuation", which you can style as a category, but one drawback is you cannot style parentheses or brackets differently than any other punctuation.

For reference, here's what is rendered currently in the demo (viewed in Chrome dev tools):

There's an opportunity to wrap commas, parens and brackets in classed <span>s for richer theming.

As CodeMirror is being re-written, perhaps there could be an opportunity to add support for themeable brackets and parentheses? I'm not sure where this feature would live, but my guess is in the language modes.

Has this been published as an npm package yet? Alternatively, is there a way to build the project to be consumed via git?

Right now, if the view is updated precisely between the point where DOM changes happen and the point where the mutation observer is called, those DOM changes are ignored. It is possible to store them and apply them after the update (mapping the change objects) though, so that's what we should do.

Right now, the horizontal scroll width is purely determined by what's currently in the viewport. So if you scroll a long line out of or into the viewport, the horizontal scrollbar might change.

Doing this 100% accurately would involve measuring all changed/new lines all the time, and is probably too expensive to be worth it. But doing something to smooth over the obvious inconsistencies from the point of view of the user would be a good idea.

Decorations should be able to influence the styling of (visual) lines somehow, either as point decorations at the start of the line, or as range decorations that influence all lines whose start they cover.

Insertions, deletions, or size changes above the visible viewport shouldn't cause text to jump around

So that there's a command available that goes back/forward to previous selections. Probably as part of the main history implementation.

Consider using Quill's Delta library. This would enable multi-user editing via operational transforms, similar to Google Docs. You could just omit the styling functionality. More info here.

Whereas, unless the widget itself defines custom event handling, it should just put the cursor next to it on the side nearest to the click.

DOM updates in this version are no longer per-line, and I was expecting big gains from that, but it seems that browsers still take a lot of time computing a layout for a long line.

One possible optimization would be to detect which part of the line is after the viewport, and replace that with a placeholder empty space node. Without line wrapping, the same approach could be used for the part before the viewport. With wrapping, that's a lot harder, since we don't know where wrapping points would be without drawing the content (but maybe text snapping into a different place when you scroll up is a reasonable price to pay for having responsive editing on a megabyte-long line).

See also codemirror/codemirror5#2090

Android Chrome deal strangely with virtual key input, and doesn't fire meaningful keydown events for the virtual keyboard. For typed text, that's okay (we can dispatch behavior on the text inserted to do things like bracket matching), but you might want to bind special behavior to enter or backspace.

When reading DOM changes on Android, we should detect changes that look like they originated with a specific key, and try dispatching that key's handlers before applying the actual change.

This involves a number of things that still have to be determined:

• What should completion sources look like (should be more flexible and less ad-hoc than the old system, support custom rendering of options, custom apply commands, sync and async operation)

• How does the UI plugin that handles completion get connected to the completion sources? Do sources act as separate plugins that expose services, or are they passed directly to the UI plugin?

• How do we structure a conventional completion UI (list below the cursor, type or arrow to select) in an accessible way?

While working on building the test suite for codemirror-ot (particularly this PR vizhub-core/codemirror-ot#4), I encountered the following runtime error when running npm test:

codemirror.next/view/src/browser.ts:11
const webkit = !ie && 'WebkitAppearance' in doc.documentElement.style
                                                                ^
TypeError: Cannot read property 'style' of undefined
    at Object.<anonymous> (/home/curran/repos/codemirror-6-experiments/packages/codemirror-6/codemirror.next/view/src/browser.ts:11:65)
    at Module._compile (internal/modules/cjs/loader.js:688:30)

While my test code is not importing anything directly from codemirror.next/view/src/browser.ts, it is loading a CommonJS build of CodeMirror (as described in #19), and the code in browser.ts ends up running at the time when anything is imported from the CommonJS build.

This issue may be premature here, as there is no official CommonJS build quite yet. But if/when a CommonJS build is introduced, this problem will crop up when folks import anything from CodeMirror in a Node.js environment, which makes sense to do for testing plugins (and server-side rendering).

Solution idea: What if browser.ts exports a function that computes and returns the information that is currently exported from that module? This would cause the browser globals to be accessed only at the time something that actually needs them is instantiated.

Alternative idea: If the build targets ES6 rather than ES5 CommonJS, browser.ts may never be loaded.

Related (CM5):

• Server side rendering codemirror/codemirror5#3701
• ReferenceError: navigator is not defined codemirror/codemirror5#5470

For things like creating multiple selections, rectangular selections, or new behavior for a given modifier + click count + button combo.

This will probably involve some kind of current mouse context being kept in the view, which lives as long as the button is held down, and influences the effect of mouse motion.

I think we'll need to move to handling mouse interaction entirely in our code, to do this properly. Have to be careful not to disrupt touch interaction.

As a curious onlooker, the first lines of code I find myself copy-pasting into a new project when trying out CodeMirror 6 are the following from demo.ts:

// FIXME these should move to commands and access the indentation
// feature through some kind of generic mechanism that allows plugins
// to advertise that they can do indentation
function crudeInsertNewlineAndIndent({state, dispatch}: EditorView): boolean {
  let indentation = (mode as any).indentation(state, state.selection.primary.from)
  if (indentation > -1)
    dispatch(state.transaction.replaceSelection("\n" + " ".repeat(indentation)).scrollIntoView())
  return true
}
function crudeIndentLine({state, dispatch}: EditorView): boolean {
  let cursor = state.selection.primary.head // FIXME doesn't indent multiple lines
  let line = state.doc.lineAt(cursor), text = line.slice(0, 100)
  let space = /^ */.exec(text)[0].length // FIXME doesn't handle tabs
  let indentation = (mode as any).indentation(state, line.start)
  if (indentation == -1) indentation = space
  let tr = state.transaction.replace(line.start, line.start + space, " ".repeat(indentation)).scrollIntoView()
  if (cursor <= line.start + space)
    tr = tr.setSelection(EditorSelection.single(line.start + indentation))
  dispatch(tr)
  return true
}

I'm opening this issue to track the task of refactoring these two functions to live outside of the demo code, so that newcomers trying out CodeMirror 6 have a "clean slate" to play with.

Thanks for all the great work so far!

I'm having a bit of trouble writing tests a new plugin, https://github.com/datavis-tech/codemirror-ot.

I'm currently writing tests that run in a Node.js environment. The issue I'm facing is that in order to test a plugin, a new instance of EditorView must be created, and the constructor for EditorView currently accesses the document global, so when running the test in a Node.js environment I get the following error:

ReferenceError: document is not defined

Here's an example test that demonstrates the issue (work in progress).

I'm not sure the best way to proceed, but I thought I'd file an issue because testing plugins in a Node.js environment is probably something many folks will want to do going forward.

Some ideas for moving forward:

• Try mocking out EditorView.
  • If the tests are written in TypeScript, a prerequisite for this would be to restructure the CodeMirror code such that EditorView is an interface rather than a class. Currently, with the restrictions posed by Typescript, there appears to be no way of mocking EditorView.
  • If the tests were written in JavaScript, it would be possible to mock EditorView.
• Try JSDom for defining document.createElement in a Node.js environment. This doesn't quite feel right, as EditorView is really designed for browsers only.
• Write tests that run in the browser, not Node.js (like the CodeMirror view tests).

Related:

• Avoid accessing browser globals at module load time #47

Will post back here if I find a suitable way forward!

Logic can stay largely the same. Will need access to token info, so this should probably wait for the highlighter integration to stabilize a bit.

In the current demo, if you type inside the const on line 2, you get strange effects (the first five chars continue to be highlighted as keyword suggesting outdated decorations, but the highlighting on the string at the end of the line remains aligned to that string).

In general, we need a more coherent strategy for dealing with preserving horizontal position during cursor motion.

We update the editor state on every input change now, rather than only on compositionend. And even if we didn't changes made through the API could come in during composition. To deal with this, the plan is to 'protect' the part of the DOM in which the composition is taking place, and only redraw that when its text content actually changes, so that things like rehighlighting don't disrupt composition.

So that you won't get empty space in your print for stuff that used to be outside of the viewport.

It would be interesting to support the use case of server-side rendering of the CodeMirror DOM.

I expect this issue to be closed as out of scope, but check this out - I actually got server rendering working in an experiment.

Here's what I had to do in order to get it to work, using JSDOM (full server file):

const dom = new JSDOM(html);   
const document = dom.window.document;
document.getSelection = () => ({}) as Selection;
const globalAny:any = global;  
globalAny.document = document;

// Ideally the stuff below would not be necessary.
globalAny.navigator = {};
globalAny.window = {           
  addEventListener: () => {}
};    
globalAny.MutationObserver = () => ({
  observe: () => {},
  takeRecords: () => {},       
  disconnect: () => {}         
}); 
globalAny.requestAnimationFrame = () => {};
const view = createView();
document.querySelector("#editor").appendChild(view.dom);       

It may be interesting and relatively easy to add a node or jsdom entry to the environments detected in browser.ts, then use that to avoid executing code that's not necessary for just constructing the DOM.

I'm struggling to figure out how to listen for changes.

Conceptually, I'd like to be able to tap into the stream of dispatched transactions (I think this is the right terminology in the new design), and also be able to emit new transactions programmatically. The context for this is integration with the Operational Transformation software ShareDB for real-time collaboration.

I'm guessing the best way to do this in the new CM6 architecture would be to create a custom plugin?

I have tried creating a custom Plugin and adding it to this experimental demo, but I'm not sure where to go next. This is what I have so far for the custom plugin:

import { Plugin } from './codemirror';
export const experimentPlugin = new Plugin({
  // Goal #1: Listen for all transactions,
  // so they can be converted to OT operations.
  //
  // Goal #2: Inject programmatically created transactions,
  // from remote OT operations.
});

I realize there's not much there (nothing at all actually), but at least I have this compiling and running. Navigating through the source code, perusing definitions of Plugin, PluginSpec, StateField, Transaction and MetaSlot is making my head spin. Also I've been looking through existing plugin definitions (such as history, keymap, and matchbrackets), looking for something similar that simply listens for transactions/events, but so far I haven't found any that seem like a good starting point.

Any suggestions would be greatly appreciated! Thank you.

If I do figure it out I'll post the solution here.

The empty space placeholder can be arbitrarily large for big documents, but browsers tend to have a limit on the height they support in a CSS property. In Firefox, the limit is 17895697 pixels. Have to test with other browsers. The library should create multiple elements below each other to work around this when necessary.

Firefox 61 on Linux.

readFile("package.json", "utf8", (err, data) => {

Takes 8 Ctrl+Right presses to reach end of line, but only 6 to return to beginning.

I'm not sure if this is a problem or to be expected. Firefox on Linux (in the github issues textarea) takes 8 to reach the end, but 10 to return to the beginning.

This is where things like tab size and line separators would go, which need a single value that can be a source of truth for various pieces of code. I assume updating options would be a kind of transaction.

It currently works per-line, which is needlessly expensive for big lines.

Embedding CodeMirror will be a bit easier with:

<code-mirror></code-mirror>

than:

<div style="" id=editor></div>

let view = (window as any).view = new EditorView(state)
document.querySelector("#editor").appendChild(view.dom)

Custom elements are natively available in Chrome, Safari, Opera and Firefox Nightly, and polyfilled in Edge and older browsers.

Right now, the gutter plugin only handles a line number gutter. It should be possible to create additional gutters and to add custom markers to them for given lines.

It currently resets the selection to a single one (after applying the inserted text at all existing selection ranges)

It might be useful to be able to turn them off

Since the view, by default, doesn't even draw them, it makes sense to disable them unless explicitly enabled by a plugin—invisible magic selections aren't a good thing

This should probably happen on the state level, since that's where most interaction with the selection takes place

So that custom code can introduce custom history events, that are applied as part of or between the history events created by editing

And see if the issues around it stubbornly showing an unclippable focus outline for a focused editable element (making a scrollable editor look awful) still exist in recent versions of Mobile Safari

Browsers don't support path searching of import specifiers - imports must be valid URLs. It looks like most of the import and exports in the codebase are relatives URLs and therefore ready to load in a browser, except for the lack of .js. file extensions.

Luckily TypeScript understands that if you import ./foo.js, it refers to the module and types generated by ./foo.ts. So you can translate this:

import {EditorState, Transaction, EditorSelection, MetaSlot} from "../../state/src"

to

import {EditorState, Transaction, EditorSelection, MetaSlot} from "../../state/src.js"

and everything will work and the output will be loadable directly by browsers.

Note: You really shouldn't need rollup for demos or tests - this individual modules will load just fine. Applications can bundle just the modules they use, if they choose to bundle.

Connecting modules to the CSS they rely on doesn't seem to be a solved problem in the JS ecosystem yet. Our plugins will often need a few lines of CSS to work. Requiring people to manually add <link> tags for each of those (which come from NPM and are probably not even in their web root by default) is a poor user experience.

I'm also not partial to schemes that inject the CSS into the page at run-time. When users are optimizing their page, they'd prefer to have control over how and when CSS is loaded, and not have that done at unpredictable moments by a script.

Who knows any existing approaches that we could take inspiration from?

They currently delete the content, because it's not rendered and hence not visible when reading the DOM. We'll either have to render an invisible placeholder, or change the way DOM text is read.

Possibly as decorations on the line boundaries. These should be rendered between the DOM elements for the lines.

The height map will have to get support for these too.

Covering the gutter will be hard to support, since that's now a separate element overlaying the content.

It would be useful to have an example with multiple buffers, similar to this CodeMirror 5 example:

https://codemirror.net/demo/buffers.html

I'm not sure if examples/demos are in scope for this repository, feel free to close if these should live elsewhere.

A generalization of this issue would be to establish a place for examples/demos to proliferate. Perhaps the current demo directory could be expanded to house multiple demos going forward, which could server as a sort of interactive test suite. Maybe this should eventually be a separate package, once #19 is tackled.

I've been experimenting in https://github.com/datavis-tech/codemirror-6-experiments, and one the things I'd like to do there is get an example with multiple buffers working. Will report back here if I do get it working eventually.

Especially in Firefox, which makes many cursor positions unreachable with the arrow keys (between uneditable elements, between an uneditable element and the start/end of line).

But there are also more general issues, like widgets that don't cover any content not being properly skipped over.

This may be a weird issue, but I was just playing with the editor on the demo page and after adding the style .CodeMirror { transform: rotate(15deg); } through Chrome dev tools, the gutter line height calculations broke.

I was thinking about this while thinking about how I could prototype a system like this projected editor I found while researching Dynamicland

Greetings,

I'm working on a demo where you can switch between views, and your previous cursor position should be scrolled into view. The scrollIntoView transaction does scroll the view, but it scrolls to the line above the line that should be scrolled into view.

For example if the cursor is on line 70, it scrolls the viewport such that the line on the bottom is line 69.

This is the snippet of code I'm using to dispatch the transaction (inspired by test-builtin-commands.ts):

const cursorPosition = view.state.selection.primary.anchor;
view.dispatch(view.state.transaction
  .setSelection(view.state.selection.constructor.single(cursorPosition))
  .scrollIntoView());

It also seems to do the same thing if I omit that second line:

view.dispatch(view.state.transaction.scrollIntoView());

To reproduce, run the demo found here in this branch: vizhub-core/codemirror-6-experiments#20

• Switch to index.js
• Put your cursor somewhere scrolled down
• Switch to another file
• Switch back to index.js
• Observe it has scrolled into view the line one less than the desired line.

When typing fast with at least one newline and immediately undoing, the editor gets into a corrupted state.

To reproduce, run the demo in /demo, bash at the keyboard with at least one newline, the immediately hit CTRL+Z.

The error that gets output:

docview.ts:383 Uncaught TypeError: Cannot read property 'domFromPos' of undefined
    at DocView.domFromPos (docview.ts:383)
    at DocView.updateSelection (docview.ts:203)
    at docview.ts:154
    at DOMObserver.withoutListening (domobserver.ts:105)
    at DocView.updateInner (docview.ts:147)
    at DocView.update (docview.ts:88)
    at editorview.ts:85
    at EditorView.withUpdating (editorview.ts:112)
    at EditorView.updateState (editorview.ts:79)
    at editorview.ts:34

This behavior does not happen if you type slowly and wait some time before hitting CTRL+Z.

As with the old specialChars and specialCharPlaceholder options. Should default to some reasonable set of characters and representation, and be configurable.

On IE and Edge, this should also take care of controlling tab size, until they implement the tabSize CSS property (feature detect) (#2)

Middle-click paste seems to generally work (in my FF), but the selection is not updated before pasting.

This could avoid flickering during throw-scrolling. It is likely to run into the ways browsers try to optimize scrolling in all kinds of ways. Should the handlers be passive? Will they even get to do the drawing in time? Have to experiment.

On modern browsers, you can do this with a css rule.

Edge/IE11 don't support sizing tabs via CSS, so on those platforms we'll have to render something else instead. For accessibility reasons, a span that contains an actual tab but has its width adjusted with CSS is probably the best approach.

babel7 released with the ability of recognize typescript like below:

https://blogs.msdn.microsoft.com/typescript/2018/08/27/typescript-and-babel-7/

Thus the type checking and compiler can be safely separated, this is good for long term maintaince of the codebase, and also good for extension, like integrating the codebase into non-ts env, using the whole babel toolkits etc.