Git Product home page Git Product logo

tap-adql-sandbox's Introduction

TAP ADQL sandbox

About

A sandbox application for executing ADQL queries via TAP interface of various data sources, such as astronomical databases. Essentially, it's just a GUI for PyVO.

TAP ADQL sandbox application window

More details in this article.

Demonstration

application-demonstration-macos.mp4

Installing

From PyPI

$ pip install tap-adql-sandbox

From sources

$ cd /path/to/repository/
$ pip install ./

or:

$ cd /path/to/repository/
$ python -m build
$ pip install ./dist/tap_adql_sandbox-VERSION-py3-none-any.whl

Running

$ tap-adql-sandbox --help

Platforms

Tested on:

  • Mac OS:
    • 11.6.4, Intel
    • 12.2.1, Apple silicon
  • Windows:
    • 10
    • 11
  • GNU/Linux:
    • Ubuntu 20.04

Known problems

Application tries to connect to remote hosts on startup and sometimes crashes

Sometimes when you are just launching the application, so you didn't even have a chance to execute any queries, you might notice that it tries to reach various remote hosts on the internet, such as obspm.fr, ietf.org or probably others.

This is because of the Astropy package, which is an indirect dependency through PyVO, which is a direct dependency of this project. Specifically, it's the hosts listed in this file. Looks harmless enough, apparently just updating some astronomical data.

Denying access to these hosts might lead to the application crash, because Astropy doesn't handle such situation properly:

AttributeError: module 'IPython.utils.io' has no attribute 'stdout'

If you get the application crashing even when access to those is allowed, try to update the Astropy (and probably also PyVO) package:

$ pip install astropy -U

I had this problem with Astropy v4.2, and it was gone after updating to Astropy v5.1. Or perhaps the problem isn't really gone, but the new package version just came with updated data, so for now there is no need for updating.

Application crashes when query has too many columns

If your query/request has a lot of columns in SELECT, the results table might not have visible contents, or the application might just crash.

Queries might fail with UnicodeDecodeError

If query results from a TAP service contain non-ASCII symbols, then PyVO will raise an exception. The application won't crash, but you won't get query results either.

3rd-party

Dependencies

  • Python 3.6 or later (the oldest tested is 3.7, though)
  • Dear PyGui - application window and UI controls
  • PyVO - handling TAP ADQL requests
  • Astroquery - looking up IDs in Simbad
  • pandas - processing results and saving to pickle
  • tabulate - printing results to stdout (with --debug)

Resources

tap-adql-sandbox's People

Contributors

retifrav avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

tap-adql-sandbox's Issues

Make first enumerating column optional

The first column that artificially enumerates result might be confusing and rather unexpected for a user:

enumerating-column

Make it optional with CLI argument and disabled by default.

Add an option for tabulate float precision

Tabulate output defaults to g(?) format, which is okay in most cases, but sometimes one might want a higher precision.
Need to add a CLI option for setting desired format for tabulate's floatfmt.

For now it will be used only in debug output, but later it will likely also apply to the query log (#11).

Closing About dialog causes segmentation fault

After About dialog is shown, if one clicks on about that..., the About Dear PyGui window will appear, but the original dialog won't be closed properly (though it will disappear from the screen). Exiting the application in this case with result in segmentation fault:

$ tap-adql-sandbox
Segmentation fault: 11

$ echo $?
139

If one opens original About dialog again and closes it using the Close button, then the application exits normally. But anyway, need to add closing call before opening About Dear PyGui window.

Add example queries for getting a list of available fields and some other examples

Users, especially those unfamiliar with SQL, will benefit from having more query examples:

  • it would be useful to have a query for:
    • all available tables
    • all available fields in a given table
  • while we are here, add some more queries examples:
    • systems with more than 5 planet: their names and count of planets
    • all the planets that have known values of mass and radius

Add more query examples

Need to add more query examples to the default library. For instance, the current NASA example is too complex, and while it is a great example for the particular purpose that it demonstrates, most of the time users would prefer to get something similar to the current exoplanets.eu example - just a simple SELECT query for one parameter and with trivial WHERE clause.

Corner tap on some Macs touchpad doesn't copy cell contents

Some Mac users (only those on ARM?) set right-click functionality on their touchpad to trigger on single click in one of the corners. In that case the copying cell content action/handler does not trigger.

I tried to reproduce it, but it works fine with corner tap on my Intel-based Mac, so the problem might be specific to ARM-based Macs.

Need to find what event is actually fired and react to that one too, or if it's impossible to catch, then perhaps just make it possible to copy cell content with double clicking - that one for sure should work fine.

Dear PyGui showcase app

Hey, it looks like you may have an interesting app for the Dear PyGui showcase. If you would like your app to be featured there, could you make a screenshot or GIF so that we can add it there?

Assigning tags to cells can result in duplicates

If results table has more than certain amount of columns and rows, tagging cells might get duplicate tag names, with results in the following exception:

Exception:
Error:     [1000]
Command:   add alias
Item:      0
Label:     Not found
Item Type: Unknown
Message:   Alias already exists

Don't hide loading indicator until the table is ready

The loading indicator hides as soon as query executes, but if query results has a lot of data, then generating the table from it can take some time, so it's too early to hide the indicator and show results block.
Need to move the hiding the indicator call to after the table is generated.

Find a better application icon

Current application icon is a bit too overloaded (and dark) for such a small picture. Need to find something with simpler imagery and colors. And preferably under an open license.

Queries might fail with UnicodeDecodeError

Some queries, such as this one:

-- https://gea.esac.esa.int/tap-server/tap
SELECT table_name, description
FROM tap_schema.tables
WHERE table_name = 'gaiadr2.dr1_neighbourhood'

might fail with the following error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 1317: ordinal not in range(128)

This is an bug in PyVO.

Multiline strings overflow cells/rows vertically

Example query:

http://voparis-tap-planeto.obspm.fr/tap
SELECT table_name, description
FROM tap_schema.tables
WHERE table_type = 'table' 
ORDER BY table_name

In the results the meteor_showers.epn_core description cell is overflown vertically, which also prevents scrolling down below it:

cell-overflown

Looks like an upstream issue. I can reproduce this with Dear PyGui 1.4.0, 1.5.0 and 1.7.1 on Mac OS and Windows.

Output query results as a table

Queries work now, but results are printed to the console (in --debug mode). Need to add one last missing thing from the major functionality: form query results into an ImGui table.

Query examples for a couple of services

Add examples for queries to exoplanet.eu and NASA databases. User can choose them from the Help menu, and they will prefill the service and query input fields.

Queries log

Would be nice to have a queries log: what queries were executed and what were results.

Right now all that is printed to stdout (in debug mode), but it might be more convenient to have a separate GUI window for this. It would be enough to just print stuff as plain text there, query results can be nicely formatted with tabulate.

Segmentation fault when SELECT has a lot of columns

With this query to http://voparis-tap-planeto.obspm.fr/tap:

SELECT *
FROM exoplanet.epn_core
WHERE star_name = 'Kepler-106'

the application crashes with segmentation fault:

$ PYTHONFAULTHANDLER=1 python -m tap_adql_sandbox
Fatal Python error: Segmentation fault

Thread 0x000070000ce9a000 (most recent call first):
<no Python frame>

Current thread 0x000000011245ee00 (most recent call first):
  File "/usr/local/lib/python3.9/site-packages/dearpygui/dearpygui.py", line 102 in start_dearpygui
  File "/Users/vasya/code/python/tap-adql-sandbox/src/tap_adql_sandbox/__main__.py", line 520 in main
  File "/Users/vasya/code/python/tap-adql-sandbox/src/tap_adql_sandbox/__main__.py", line 526 in <module>
  File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87 in _run_code
  File "/usr/local/Cellar/[email protected]/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197 in _run_module_as_main
Segmentation fault: 11

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.