QEP 51: Generic Helping System for QGIS about qgis-enhancement-proposals HOT 22 OPEN

I don't think expression help should be included here. It has quite different requirements then static documentation. For instance, I intend to (hopefully soon) add support for hovering over function names in an expression to see a summary of the function's syntax. This is straightforward with the current structured json approach to expression help, but much more work + duplication if expression help is moved back to static docs.

from qgis-enhancement-proposals.

Hi Nyall,

On Fri, 06. Nov 2015 at 11:51:34 -0800, Nyall Dawson wrote:

I don't think expression help should be included here. It has quite different
requirements then static documentation. For instance, I intend to (hopefully
soon) add support for hovering over function names in an expression to see a
summary of the function's syntax. This is straightforward with the current
structured json approach to expression help, but much more work + duplication
if expression help is moved back to static docs.

That might also apply to processing algorithms. I don't know that much about
processing, but I imaging that the parameters, inputs and outputs also have
much in common (like operators, operand and functions in expressions) and
would also benefit from more structure like in the expression help when
it comes to translation.

There is probably a lot of duplicated content (eg. same parameters descriptions
in multiple algorithms) that would have to be translated separately, when we
start translating the full pages (like now in context help
or the expression help prior to 82b34af).

Jürgen

Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31
Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50
Software Engineer D-26506 Norden http://www.norbit.de
QGIS release manager (PSC) Germany IRC: jef on FreeNode

from qgis-enhancement-proposals.

@nyalldawson @jef-n

I understand the need/wish to have a structured data structure, if you want to start stuff like auto-completion etc. And hovering (needing FAST access to strings) would indeed have access to the strings in front. But I think keeping documentation systems translatable and maintainable is more important then hovering functionality, also because that hovering should be translated/translatable too.

So what about this plan:

• expression (and processing?) work with a json structure to be able to check type, show (english) operands and operators, and maybe some kind auto complete, but this structure also has and uses a link (or key) to the generic system to show (translated) descriptions or further info (like good examples or graphics). So all TEXT (not talking about parameter short texts here) will come from help.
  This would maybe make your hovering functionality less descriptive, but keeps the help maintainable?

Compromise? Or do I miss your points.

from qgis-enhancement-proposals.

Well.. I fail to see how that's a better approach then what we currently have. Right now, we've got a process where expression help is structured and separated from the actual presentation of the help. This allows us to maintain consistency across all function help, alter the presentation in one place and have all help update, easier translation (in that it does not require retranslation if the html format changes). Moving back to static files would be a real step back here, both from a code, user AND translator's perspective.

What about just having a script that generates static (already translated) expression help for the docs? Potentially they could even be moved to a git submodule or something if that helps...

from qgis-enhancement-proposals.

@nyalldawson I fully agree that having the help in one place is the best way, and I hope that a developer when he introduces an expression of function takes some time to write an good description.

But my experience is that these descriptions are short and cannot contain images.
While specially for the spatial ones, one or more images say more then 1000 words.
See for example our expression help for 'difference' (Returns a geometry that represents that part of geometry_a that does not intersect with geometry_b.) and the one that our brother has:
http://resources.arcgis.com/EN/HELP/MAIN/10.1/index.html#//00080000000r000000

THAT is the sort of help I'm hoping for :-)

Also because this texts about 'difference' is not only used in expression but also in algorithms: let's provide one place where this is written in a way that everybody understands.

Another argument to write separate docs is that the 'fixme' link in the footer makes it easy for users(!) to change/make pr when the description is not clear enough. Dev's often write a help string when they do implementation, and after that it is never touched or updated again (see context help examples)

My last argument is that I want to minimize the amount of systems and scripts that we have to automagically create all kind of docs/translation files et.

from qgis-enhancement-proposals.

What about being able to use rst inside the json structure?

from qgis-enhancement-proposals.

@m-kuhn this does not make it easier for non-devs to change/write/update the help. IF the expression already has a link to an (even empty) help page in github, a doc writer can writer further help

@nyalldawson can you point to some working example you have in place now?

from qgis-enhancement-proposals.

We could include a link to the source on github to edit. That would be easy enough I would say.

https://github.com/qgis/QGIS/blob/master/resources/function_help/json/difference

from qgis-enhancement-proposals.

@rduivenvoorde would that work for you?

name: difference
type: function
description: |
  Base functionality
  ------------------

  Returns a geometry that represents that part of geometry_a that
  does not intersect with geometry_b.

  Null geometries
  ---------------

  If any of the two geometries is empty, the result will be empty
  as well.

  .. image:: difference.png
     :alt: Picture that illustrates the difference function

arguments:
- arg: geometry_a
  description: a geometry
- arg: geometry_b
  description: a geometry
examples:
- expression: "geom_to_wkt( difference( geom_from_wkt( 'LINESTRING(3 3, 4 4, 5 5)' ), geom_from_wkt( 'LINESTRING(3 3, 4 4)' ) ) )"
  returns: LINESTRING(4 4, 5 5)

from qgis-enhancement-proposals.

Are there any work on this?

from qgis-enhancement-proposals.

none that i know of

from qgis-enhancement-proposals.

I once did a small 'analysis' of current context help:

http://hub.qgis.org/wiki/quantum-gis/ContextHelp

from qgis-enhancement-proposals.

This is a quick setup I once did to show that using a web-widget linking to a deep link in QGIS documentation should work:

ContextHelp.zip

from qgis-enhancement-proposals.

We also should think about using existing help/documentation for 3rd party tools, e.g GRASS/GDAL/SAGA providers in Processing. This documentation hosted on separate sites so we need a mechanism to open this pages too

from qgis-enhancement-proposals.

Do you want to directly link to these external pages from within QGIS?
Maybe sometimes it would be nice to have a description of details about the QGIS integration. So

• a simple page with Algorithm name + external link (by default)
• a handwritten page + external link (where explicitly specified)

could be another approach.

from qgis-enhancement-proposals.

Do you want to directly link to these external pages from within QGIS?

Why not? If documentation already written by someone else, personally I see no sense in recreating or copying it. Maybe for such cases even

a simple page with Algorithm name + external link (by default)

will be an overhead, as instead of single click user will need two click to access documentation.

Of course, if there are some differences in implementation or additional details about QGIS integration, we should create page with this information and link original docs from it, as you suggested.

from qgis-enhancement-proposals.

Seems the most tricky part in this QEP is creation of mapping between help URL and key, as described in the original post

one config file (JSON, YAML?) (some redirection) containing:

• key (preferably recognizable, like: processing_algorithme_grass_whatever )
• uri ( which an be expanded to either a url or a file into a zip (zip://......)
• way of opening (defaulting to …?)
• ???

from qgis-enhancement-proposals.

personally I see no sense in recreating or copying it

Same here.

What I was thinking was that when linking directly to external pages we can never be sure that the URL of the upstream doc doesn't change. Maybe we could have our server send a HTTP 30x (redirect) code or a <meta http-equiv="refresh"...>. Then we still have the possibility to remove the redirect and show our own page at any time or route to a different page without changes to the client app.

Another question is, if we need these translated as well at some point and how/where this is done (in our docs/in the upstream docs, how does the infrastructure play together) or available offline.

creation of mapping between help URL and key

We could just have the relative path to the file as key. So e.g. processing_algs/qgis/vector_general_tools.html#delete-duplicate-geometries.

We can still add an additional "alias" file somewhere later to make the keys simpler at a later point.

uri and way of opening should probably not be part of a config file but runtime parameters (checks for local zip file availability, language, embedded viewer...). Or I did not understand.

For expression and processing help, I would like to re-propose the above yaml-based structure, maintained in the main QGIS repository so we keep the possibility to allow things like auto-completion without trading this for read-, maintain- and translatability. We can add some webhook or travis script, that creates a pull request against the docs repo whenever the code has been updated.

• documentation can be an integral part of a pull request against expressions/processing.
• documentation can be provided in a formatted way
• a default documentation can be delivered without downloading the whole documentation

This would also integrate with the requirement to link to the upstream docs

name: v.clean
reference: https://grass.osgeo.org/grass73/manuals/v.clean.html
# descriptions: | # If no description is provided, the doc will be redirected

from qgis-enhancement-proposals.

I'm fine with yaml-based help for expressions and Processing algs when this makes sense.

Regarding implementation, IMHO we need:

• Introduce new QGIS settings:
  • location of local documentation (single path to the root directory), by default set to some predefined place, e.g. $HOME/.qgis3/docs. We should also allow defining intranet end-points here (useful for closed networks)
  • mode selector, where user can chose how documentaion should be opened, e.g. try only local docs, try only online docs, try online and fallback to local if not available, try local and fallback to online if not available
• develop new singleton(?) class QgsHelp which will construct full documentation URI based on the given "key" and QGIS settings and UI language. As "keys" a relative path to the corresponding file will be used, e.g. processing_algs/qgis/vector_general_tools.html#delete-duplicate-geometries.

QgsHelp class should take in account language of the QGIS UI and fallback to English version if such language does not exists. If no help found, return URI to the placeholder page, e.g. "Sorry, documentation is not available".

Constructed documentation URI should be opened in system default web-browser.

from qgis-enhancement-proposals.

Sounds good to me.

We could have an "array" of url prefixes, then order of precendence, additional endpoints etc wouldn't be a trouble. If one of the resources returns something else than 200 or 30x (HTTP) or the file is not found, the next resource will be tried. We could also add our own schema prefixes (like zip:// etc.) if there is a need for it.

QgsHelp class should take in account language of the QGIS UI and fallback to English version if such language does not exists.

For webbased content, it could also just always request the localized version and the server will transparently deliver a localized one, I think that's the easiest solution. I think it's possible that even with downloaded files, we will always have each file in the docs folder if it's only partially or completely untranslated.

from qgis-enhancement-proposals.

@m-kuhn @jef-n @nyalldawson so we agree that adding a structure to the processing algo's/expressions/functions is valuable and could also lead to more efficient translating?

But can we also agree that this structure (be it JSON, YAML or Yet Another Format...) should only contain a minimal description (which can be a starting point) for the real docs? So no need to add pictures or rst/markdown here... Just add an url to the structure?

About configuration file/list of the keys: could that be one (ini?) file with some sections in it?
Thinking about sections for: manual (for current help button helps), expressions, algorithms, grass, saga ??? So we can have different ways of handling the url's: I'm not sure if grass and saga will make their docs easily downloadable for us.
Section also would make it possible to create scripts for certain sections to recreate this config and the (placeholders in) docs based on the sources?

The URL's could maybe have parameters in it like:
$HOST/$PATH/$LANG/processing_algs/qgis/vector_general_tools.html#delete-duplicate-geometries
So based on the section logic can create the final uri's based on the section paramaters (like base-url, zipfile location etc)

from qgis-enhancement-proposals.

@rduivenvoorde can you add an example of what you propose to be in the config file/some example structure?
I cannot follow if this file should be on the client- or server-side and what exactly is defined in there.

Can you also shortly explain why you want to have only a minimal description without any images in the help?

from qgis-enhancement-proposals.