Follow-up issue to #102

When expanded, the sidebar goes all the way to the left edge of the screen:

When hidden, there is a (slightly darker) gutter of ~2 left behind:

I have no idea why? There doesn't seem to be a margin on the containers that would do this?

• split at semicolons
• only execute query at cursor
• if selected more than 1 query, return results in tabs

Could be enhanced with #131 ; create a "Run All" button or checkbox? See also #155

Hi. I encountered two problems when I use harlequin on Windows:

1. I can't enter even one line of code smoothly. There seems to be a problem with the formatting.
2. Sometimes the editor exits abnormally. For example, pressing the Ctrl when there is no content.

ctrl+? doesn't work in the terminal, it's an alias for backspace (?). F1 should work!

Need to design this so it doesn't take up too much real estate.

DuckDB has support for attaching multiple databases. The catalog UI currently only takes the default DB into account. It would be great to show all databases/catalogs and their schemas on the left hand side, with the current default database highlighted.

https://duckdb.org/docs/extensions/json.html

json_serialize_sql()

Main Actions:

• #31
• ctrl + enter = run query
• #94
• ctrl + s = save
• ctrl + o = open
• #20
• #149

Navigating and Editing:

• ctrl + home = home home
• ctrl + end = end end
• #21
• ctrl + x = cut
• #22
• ctrl + v = paste
• #30
• tconbeer/textual-textarea#77
• F2 = focus on text editor
• F5 = focus on the results viewer
• F6 = focus on the data catalog

Tabbed editor (multiple buffers):

• #23
• ctrl + w = close tab
• ctrl + j = prev tab (not doing; ctrl+j is alias for ctrl+enter to run query)
• ctrl + k = next tab

Undo/Redo:

• tconbeer/textual-textarea#12
• ctrl + y = redo

Find/Replace:

• tconbeer/textual-textarea#11
• ctrl + h = replace

Goto:

• tconbeer/textual-textarea#10
• tconbeer/textual-textarea#9
• tconbeer/textual-textarea#90

Change Layout:

• #29
• #100

Not doing (key events not received):

• #24
• alt + down = swap line down

It's one widget, so will need to change the tooltip based on mouse location. Maybe better with a timeout so we only compute that for longer hover?

Limit checkbox is all screwed up

Simon has a great tutorial here:
https://til.simonwillison.net/homebrew/packaging-python-cli-for-homebrew
and also here:
https://til.simonwillison.net/homebrew/auto-formulas-github-actions

Hey @tconbeer, very cool and useful project.

Running this is a MacOS default terminal, all commands work except Run Query.

I tried all combinations (cmd + enter, ctrl + enter), but it results in a new line.

Any thoughts on what could be the issue?

Saving a query with CTRL+S is super useful, CTRL+E to export query results to a CSV would be very helpful as well

Shouldn't be hard to store the editor's text property in a file.

Restore from cache on open, probably after we create the duckdb connection.

See #71

Might fit when we implement #35 and/or roll a custom footer.

hopefully we can just push a new screen and then mount the current widget onto it.

F11 is intercepted by windows terminal, and doesn't come through as a key event; let's do f12 f10 instead.

Test thread worker vs the cli, figure out if multiprocessing would use more cores etc

The real bottleneck is loading lots of data into the table; not sure what to do about that.

Harlequin looks good in WSL, but not native in Windows, where the terminal displays more colors:

In particular, the background colors for the text editor (at least monokai theme) don't match the background colors for the data catalog and results viewer; the run query bar has a totally different (bluish) background.

Currently Harlequin's query editor only supports pasting from text that has been already copied within the query editor. A common use case for SQL editors is copying SQL snnippets from the system clipboard, so this should be supported.

I can see from the doc string of the on_paste function in tui/components/textarea.py that this is a known limitation, but thought I'd create an issue to save others trying to work out if this is a bug or known limitation.

When I initiate a paste event I can see that the event param passed to the on_paste function has my copied text from the system clipboard in its text attribute, so hopefully that means all the ingredients are there ready to add this feature.

Need to wait longer before checking data after running the query.

alt+up doesn't generate a key press I can listen for. It turns into this?

Key(
    key='escape',
    character=None,
    name='escape',
    is_printable=False,
    aliases=[
        'escape',
        'ctrl+left_square_brace'
    ]
)
Key(key='up', character=None, name='up', is_printable=False)

Being developed here: https://github.com/tconbeer/textual-textarea

https://rich.readthedocs.io/en/stable/syntax.html#theme

Harlequin with latest duckdb on fedora 38.
Is it possible to run a selection from an sql scirpt file rather than the whole file?
Apologies if I am missing somehting very obvious.

I dropped this in the migration to textual_textarea; needs to be reimplemented

Issue
Opening a database with --readonly should allow harlequin to open database files that are currently opened by another process for writing. Right now, harlequin attempts to lock a DB file, even when the --readonly flag is passed, resulting in an error when attempting to open a database that is concurrently open by another process.

❯ harlequin -r /tmp/mimo.db
╭─ DuckDB couldn't connect to your database. ───────────────────────────────────────────╮
│ IO Error: Could not set lock on file "/tmp/mimo.db": Resource temporarily unavailable │
╰────────────────────────────────────────────────────────────────────────── Try again? ─╯

Background
Currently the DuckDB driver allows a single writer and multiple readers to concurrently work on a single database file; see duckdb/duckdb#1330. Therefore, harlequin should open be able to open a database in 'reader' mode that is concurrently open by another process in writer mode.

Reproduction

1. Open a DuckDB database file in writer mode with harlequin harlequin /tmp/mimo.db
2. In another window open the same database in readonly mode harlequin -r /tmp/mimo.db
3. The second command will fail to open the same database in read only mode.
4. Kill the first harlequin process
5. Running harlequin -r /tmp/mimo.db will now work fine

So, first, sorry about the PR spam on all your repos the last few days 😅

The reason I was doing that was because I've been on a personal education journey to learn both DuckDB (because I resonate w/ SQL much more than pandas) and PRQL because it's such a powerful and expressive alternative to raw SQL.

So if you look at my fork on the prql branch, I've cobbled together a janky version of this based on all my PR's to this repo and textual-textarea.

Granted that branch is definitely not ready for prime-time, but the bones are there and it actually works pretty great.

The meat of the changes are this:

1. ℹ️ This functionality is based on this DuckDB extension: https://github.com/ywelsch/duckdb-prql
   1. PRQL works by getting compiled directly to SQL
   2. PRQL already has a duckdb dialect target built-in
   3. This plugin works by running the PRQL compiler in duckdb dialect mode on-the-fly
   4. Because of this, no changes are needed to the Python API outside of making sure the plugin is installed and loaded before passing PRQL strings to the sql() method on the connection.
2. ℹ️ The change to textarea was just a convenience for me so that I could have some "init" scripts in my home dir that were easy to load
3. ⚠️ Unfortunately, this extension does require that unsigned extensions be enabled, hence the PR for the -unsigned flag.
4. ⚠️ This extension has its own release cadence, and doesn't have a 0.8.1 release yet, which is why on my personal fork I've pinned duckdb to 0.8.0 in the pyproject.toml.
   1. This is an artifact of the URL convention that duckdb uses for extension downloads even though semver kinda indicates that the extensions should remain compatible
   2. I think the fix to this would be vendoring the plugin but that would drastically complicate build and delivery of this project so I'm not sure the way you'd want to deal with that if this went forward.

Obviously, there's a lot from the above that raises questions about if this feature is worth it. But if you think it's something that would be valuable I'm more than happy to continue assisting with contributions.

And there would be more work down the road to make "prql mode" more of a 1st class citizen like including syntax support.

So I understand if it's not something you're interested in including, and it's pretty trivial to keep a patch for this going. Also, if we merge the -unsigned changes, then I don't even need the code patches, because you can load prql using regular plugin load statements.

Thanks in advance for looking at this!

Will need to upgrade to duckdb 0.8.1.

It might work out of the box, if you first set the motherduck_token env var.
https://motherduck.com/docs/authenticating-to-motherduck#using-the-service-token-to-connect

Should add a CLI option for the token and docs example.

May as well add an option for SaaS mode while we're at it.

https://github.com/duckdb/duckdb-web

The built-in widget is ok, but its data structures are super inefficient, which limits us to about 10k rows. It shouldn't be too bad(?) to fork the current widget and represent each row as a tuple of its data.

Need to figure out how to handle the api/output of explain. IT's a 2-column query; the second column has multiline output, which the data table doesn't handle well. Maybe somehow detect explain and use markdown instead?

At some point the scrollbar for the query editor went back to the default colors -- our CSS stopped working.

Heya!

Was just playing around and input a query I knew to be bad, and managed to crash harlequin:

select 0::struct(id int)

This was in version 0.0.8

I was trying to use this inside of my nba-monte-carlo project.

I opened the codespace, ran pip install harlequin followed by cd data/data_catalog/. If I think try to open my db with harlequin mdsbox.db, I get the following error:

Traceback (most recent call last):
  File "/home/vscode/.local/bin/harlequin", line 5, in <module>
    from harlequin.cli import harlequin
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/__init__.py", line 1, in <module>
    from harlequin.tui.app import Harlequin as Harlequin
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/tui/app.py", line 13, in <module>
    from harlequin.tui.components import (
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/tui/components/__init__.py", line 1, in <module>
    from harlequin.tui.components.code_editor import CodeEditor as CodeEditor
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/tui/components/code_editor.py", line 4, in <module>
    from harlequin.tui.components.textarea import TextArea, TextInput
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/tui/components/textarea.py", line 16, in <module>
    from harlequin.tui.components.error_modal import ErrorModal
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/tui/components/error_modal.py", line 7, in <module>
    class ErrorModal(ModalScreen):
  File "/home/vscode/.local/lib/python3.9/site-packages/harlequin/tui/components/error_modal.py", line 13, in ErrorModal
    name: str | None = None,
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Follow-up to #80

right now setting a theme just impacts the query editor pane, and it looks bad for most themes:

We could carry the theme colors throughout, with a lot of work.

Unfortunately, these are Pygments themes, which don't line up for Textual's planned support for themes (e.g., light/dark etc.)

That was 12 versions ago (and counting)!