OASP still occurs in our scripts as OASP_PROJECT_HOME:
https://github.com/devonfw/devon-ide/blob/199ba7ab931e293fd8c8629ef2535b40622d36e7/scripts/src/main/resources/scripts/environment-project#L6

We should get rid of all this legacy stuff and clean up. Maybe we do not even need the variable (see #5).

Logfiles should not be commited to git:
https://github.com/devonfw/devon-ide/blob/master/configurator/Configurator.log
File needs to be deleted from git and pattern added to .gitignore

Eclipse m2e is for some reason unable to find maven archetypes automatically from maven central. To make archetype creation work properly in eclipse using m2e you need to configure an archetype catalog. devon-ide should do that automatically so you can create a new devon project easily in devon ide.

Findbugs is dead, long live SpotBugs.
https://marketplace.eclipse.org/content/spotbugs-eclipse-plugin

So lets replace it accordingly.

Before #31 and #21 we had different releases per OS for our scripts package (from oasp4j-ide). Now this has changed and there is just one *.tar.gz release.
Hence we should make that the main artifact and ommit the classifier (assemblyId that was all, windows, linux before).

The new devon-ide is mainly based on bash scripts. For windows devon.bat and environment-project.bat only do the fundamental equivalent to setup the environment but will delegate everything else to the "bash version" of devon CLI. Therefore devon.bat needs to call devon in bash. On Windows there are several variants and possibilities to get a bash:

• cygwin
• git-bash
• unxutils
• Win10 is about to ship an update that will bring bash natively to all Windows installations with the next update.

Furhter some of the above variants may differe between 32 and 64 bit versions.
The devon.bat script shall do its best to support as many variants as possible to get the highest chances for success and best UX.

Expected behaviour would be to open the Terminal and cd to the selected folder without setting up IDE (as there is none).

java.lang.NullPointerException
                at com.sun.org.apache.xerces.internal.dom.ElementImpl.setAttributeNS(ElementImpl.java:645)
                at com.devonfw.ide.configurator.merge.XmlMerger.merge(XmlMerger.java:115)
                at com.devonfw.ide.configurator.merge.XmlMerger.merge(XmlMerger.java:85)
                at com.devonfw.ide.configurator.merge.XmlMerger.merge(XmlMerger.java:80)
                at com.devonfw.ide.configurator.merge.XmlMerger.merge(XmlMerger.java:70)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:65)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.Configurator.createOrUpdateWorkspace(Configurator.java:80)
                at com.devonfw.ide.configurator.Configurator.run(Configurator.java:153)
                at com.devonfw.ide.configurator.Configurator.main(Configurator.java:189)

Too bad...

We need to figure out if an eclipse workspace is locked or not

• https://bugs.eclipse.org/bugs/show_bug.cgi?id=197428
• https://stackoverflow.com/questions/33431762/how-to-check-if-eclipse-workspace-is-locked/33432112?noredirect=1#comment96829310_33432112
• https://stackoverflow.com/questions/14266016/why-doesnt-filechannel-trylock-prevent-access-across-multiple-processes

As we are heavily going to depend on thrid party under different licenses we need to take some care about licensing and terms of us. This IMHO implies the following aspects:

• Update TERMS_OF_USE to reflect this including a list of all third party with their licenses/terms
• Request an explicit user confirmation on the first usage of devon-ide

I have integrated devon4j template into devon-ide. You only type devon java create com.example.my-project to create the project my-project with groupId and package of com.example. We need the same for other stacks such as angular (devon4ng).

Due to the fix of #27 I broke the windows support unfortunately.
When devon is called with additional params in CMD then the BAT script delegates to the devon bash scripts in git-bash. As the variables from windows CMD are then passed to bash also DEVON_OLD_PATH remains set. This causes the devon bash script to reset the PATH to DEVON_OLD_PATH what unsets the "unix" path in bash so dirname, basename, cp, mkdir, etc. are all no longer available and scripts fail.

SEVERE: Configurator failed: Failed to copy file workspaces/main/devon-ide/settings/src/main/settings/eclipse/workspace/update/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.m2e.core.prefs.bak to /projects/devon/workspaces/main/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.m2e.core.prefs.bak
java.lang.IllegalStateException: Failed to copy file workspaces/main/devon-ide/settings/src/main/settings/eclipse/workspace/update/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.m2e.core.prefs.bak to /projects/devon/workspaces/main/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.m2e.core.prefs.bak
                at com.devonfw.ide.configurator.merge.FallbackMerger.copy(FallbackMerger.java:38)
                at com.devonfw.ide.configurator.merge.FallbackMerger.merge(FallbackMerger.java:20)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:65)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.merge.DirectoryMerger.merge(DirectoryMerger.java:69)
                at com.devonfw.ide.configurator.Configurator.createOrUpdateWorkspace(Configurator.java:80)
                at com.devonfw.ide.configurator.Configurator.run(Configurator.java:153)
                at com.devonfw.ide.configurator.Configurator.main(Configurator.java:189)
Caused by: java.nio.file.FileAlreadyExistsException: /projects/devon/workspaces/main/.metadata/.plugins/org.eclipse.core.runtime/.settings/org.eclipse.m2e.core.prefs.bak
                at sun.nio.fs.UnixCopyFile.copy(UnixCopyFile.java:551)
                at sun.nio.fs.UnixFileSystemProvider.copy(UnixFileSystemProvider.java:253)
                at java.nio.file.Files.copy(Files.java:1274)
                at com.devonfw.ide.configurator.merge.FallbackMerger.copy(FallbackMerger.java:36)
                ... 10 more

So it seems that the copy command does not allow overwriting files by default and that needs to be fixed asap!

With oasp4j-ide we could just deliver a zip with the software folder included with e.g. Commander included. We can still do that with devon-ide but the cool thing is its new lightweight approach to download required or requested software on demand. This also allows to get the right software for your current OS and make the solution portable. Even further this approach allows upgrading to newer versions much easier.
So should we create a commandlet for ConEmu and/or Commander?

Do we still require or suggest gulp? Then we should add a gulp commandlet.

We need to agree on how we want to have console support in the devon-ide. Please considere there are at least the following candidates:

• windows
  • cmd
  • power shell
  • git-bash
  • cygwin bash
  • windows bash (https://www.computerbild.de/artikel/cb-Tipps-Software-Windows-10-Linux-Bash-aktivieren-16199409.html)
  • all the above in ConEmu
  • all the above in Commander
• mac
  • terminal
  • iterm2
• linux
  • xterm
  • gnome-terminal
  • terminator
  • konsole

A general question is of course: Do we need a CLI command to open a terminal if that CLI command is to be started from a terminal?

However, on windows it would be nice to run devon bash from cmd or power-shell to open a bash on windows and let devon-ide autodected the first installed option providing a bash. Also we could easily create launch scripts in the toplevel directory during setup that you can double click to get the desired console (see #60).
Again to question this: We already have even better solutions to integrate this into the OS file browser (windows explorer, MacOS finder/commander one, nothing yet for linux). Should we automatically install this during setup and forget about console scripts at all?
WDYT?

see https://github.com/devonfw/devon-ide/wiki/Redesign

Our checks for existence of files or directories in bash are always like this:

if [ ! -d $SOFTWARE_PATH ]; then

The variable has to be quoted otherwise this check is wrong. If the variable is undefined, the if will not evaluate to true.
This is also the reason why #12 did not cause an abortion of the workspace setup even though the variable was not defined.

We already have a ng commandlet for angular-cli. However, without node.js support the user needs to manually install node.js into his system/path. This is not consistent with our approach and expected UX of devon-ide.

Hence we need to add a node commandlet. This whould then be able to download and install node.js. We need to check if we want to use node version manager or manage the node version on our own.

Category: Bug

Severity: Low

Expected behaviour: No ERROR level logs during mvn install

Actual/current behaviour: Several messages like

[ERROR] OS=Windows and the assembly descriptor contains a *nix-specific root-relative-reference (starting with slash) /

Steps to reproduce: Run mvn install in fresh cloned devon-ide

Description of issue / expected enhacement /Comments:
see https://stackoverflow.com/questions/32604733/error-os-windows-and-the-assembly-descriptor-contains-a-nix-specific-root-relat

Your environment - windows/Linux, Devonfw version , component:
devon-3.0.0-SNAPSHOT
Windows, Apache Maven 3.5.4, Java version: 1.8.0_181, vendor: Oracle Corporation

Wiki documentation has been entirely rewritten and is now detailed and improved.
We should be able to generate it as PDF.
This generation should be included into the build process and PDF should be part of the scripts package.

This comment is copied from windows script:
https://github.com/devonfw/devon-ide/blob/02b7493440e987fdfcc516a67e754807dcb8b412/scripts/src/main/resources/variables#L3
It needs to adopted to linux/macOS

It will be a big thing but we definitely need to integrate CobiGen into devon-ide and our new CLI. I leave this issue to the experts from the CobiGen team...

The locations of HOME (~) and Downlods (~/Downloads) sometimes differ leading to duplicated downloads, devon script installation and licenses agreement. Locations are:

• C:\Users\«user»\ for windows and git-bash on win
• C:\cygwin[64]\home\«user»\ for cygwin
• /Users/«user»/ on mac
• /home/«user»/ on linux
• Further if you are using virtualization e.g. VM Ware Fusion can "embed" your native Downloads directory from your host into the VM. So I am running windows 10 as VM inside MacOS. Then in Win10 I see my MacOS "Downloads" inside Windows Explorer. But it is redirecting to \\vmware-host\Shared Folders\Downloads and still there is a regular folder C:\Users\«user»\Downloads where devon-ide will put Downloads from git-bash.

This is surely nice to have. Maybe we should not bother too much about cygwin but if we want to support both git-bash and cygwin, then IMHO it would be good to only define a single location .devon and Downloads on the same machine.

With our commandlet infrastructure it is now easy to support jenkins. We should add it for local CI on e.g. git file URLs so local commits get build and tested. Once jenkins success notification is there you can push without danger of breaking stuff.

Due to #34 we want to be clean on licensing and legal terms.
One 3rd party item is currently ObjectAid that has this license:
https://www.objectaid.com/license-terms
cite:

You must have a valid license to use ObjectAid software. Licenses are for individual users or computers; they are not shareable floating licenses. The ObjectAid Class Diagram is still free to use and does not require a license.

From a legal / lawyers point of view this passage is dangerous and can lead to trouble. The "ObjectAid Class Diagram" is also "ObjectAid software". Hence it would require a valid license even though it is "still free to use and does not require a license". IANAL but after all the discussions with legel departments I can only see trouble here.

IMHO it is a pitty as ObjectAid is nice (and I personally have bought a license to also use the other features), but to stay on the safe side, I will rather remove it from our defaults.

For platform independence we (and all projects using devon-ide) currently have to provide and maintain variables "scripts" together with a redundant windows variant variables.bat.

With oasp4j-ide the support for platforms other than Windows was extremely poor. Therefore projects so far only cared about windows scripts (variables.bat, ide-properties.bat, variables-customized.bat). With the new devon-ide and cross-platform support (see #21) all this changed.

Now, it would be awesome to find a neutral way to tweak variables. Currently the variables are sourced and variables.bat are executed. This is pragmatic and allows to even unset variables or do additional initialization logic.
However, as some of these files come from external sources (git repositories) there is also the danger that some evil attacker could add script commands to do harm.

JSON or XML is most probably not the best formats to be parsed in bash or CMD but it would be nice to have an option to define the variables in an OS independent and neutral way. Still something like Java properties files would be an option if we could find a simple way to "parse" them in Bash and CMD without executing code. We have to be aware that we do not only declare flat properties but also arrays (ECLIPSE_PLUGINS) but we could also modify the eclipse commandlet to parse a comma or whitespace seprated string.

https://github.com/devonfw/devon-ide/blob/02b7493440e987fdfcc516a67e754807dcb8b412/scripts/src/main/resources/create-or-update-workspace#L44

Can never ever have worked like this...

• Bump release version, commit and tag
• Create and deploy to OSSRH and finally maven central
• open next SNAPSHOT and push changes and tags

From Yammer I got the feedback that the latest devon-dist-current.zip is causing an error running create-or-update-workspace.bat or update-all-workspaces.bat.
This is the log outout:

jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.Configurator logCall
INFO: com.devonfw.ide.configurator.Configurator -u
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log err
SEVERE: No workspace folder configured.
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log info
INFO: USAGE: [-v <variables-file>] -w <workspace-folder> -t <templates-folder> -u|-i
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log info
INFO:   -v <variables-file>:   specifies the properties file to use for replacements of variables in templates.
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log info
INFO:   -w <workspace-folder>: specifies the folder containing the workspace to manage.
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log info
INFO:   -t <templates-folder>: specifies the folder containing the templates to setup and update the workspace.
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log info
INFO:   -u:                    operation to create or update the workspace.
jan 08, 2019 11:16:35 AM com.devonfw.ide.configurator.logging.Log info
INFO:   -i:                    operation to do the inverse logic and map back the workspace changes into the update templates.

When I just download and extract devon-ide from the scripts package, I have to manually add my personal settings and then run setup. However, if no settings are present, then the setup will download the default settings and use them. This is better than nothing (and fine for testing) but not what we want for a professional project. Therefore we should add the following feature:

• if no settings are present on setup, the user shall be asked for a settings URL (typically a git URL). If provided by the user that settings will be downloaded/cloned. From these settings all the further setup shall be configured so the project settings can override and define the tools and tool versions to install
• if the user does not want to provide a settings URL, the current default may still apply
• if the setup script is invoked with the settings URL as argument there should not be an interactive question but that settings URL shall directly be used.
• if the settings URL is not a Git Repo but a download URL to an archive (e.g. zip) then a new git repo shall be initialized and the archive shall be added to that repo. Having version control for settings is a recommended feature in any case.

We have created migration support in devcon:
devonfw-forge/devcon#135

This feature shall be integrated in devon-ide. IMHO devcon is to be considered dead on the long run. I would expect devon-ide to be the new solution as a CLI tool. For advanced UX plans are already discussed to create a new devon-dashboard. This could be part of devon-ide or could be a separate add-on. This dashboard could inform about notifications and could also offer all CLI use-cases of devon-ide in a graphical user-interface.

For the integration of the code migration we should discuss the design:

• shall this code simply get integrated into our configurator?
• or shall we create a separate tool (as JAR) for that to keep the configurator isolated?

I currently tend for the isolation.

Linux script is requiring the variable ECLIPSE_REPLACEMENT_PATTERNS_PATH to point to the replacement properties:
https://github.com/devonfw/devon-ide/blob/199ba7ab931e293fd8c8629ef2535b40622d36e7/scripts/src/main/resources/create-or-update-workspace#L54

However, no such variable is ever defined.
The correct variable (like on Windows) is REPLACEMENT_PATTERNS_PATH defined here:
https://github.com/devonfw/devon-ide/blob/master/settings/src/main/settings/ide-properties

Historically we needed an application server or servlet container to run our apps. With the move to spring-boot this is no more required and even discouraged.
For running tomcat in Eclipse we had preconfigured everything in our IDE:

• software package was containing tomcat
• workspace was containing a preconfigured Servers folder
• workspace configuration contains WTP settings for tomcat.

We can get rid of all this stuff as it is only causing confusion and is even anoying for some users. Individual projects thay may still need it, can of course still keep it and will typically not be affected by our settings. Only new projects starting from scratch can be affected and if they really need it they can still get it from git history, community request, or from other projects...

See devonfw/docgen#2
The pom.xml in wiki needs to be updated after the above issue is fixed.

Then we can proceed with #43

We have done testing on MacOS now. Also Linux is getting stable.
For the future we should therefore rework our OS support:

• IMHO we should also release a MacOS bundle as currently MacOS users need to know that they have to download the linux distribution.
• IMHO we should have separate documentation per OS for "installation and setup" of the solution.
• IMHO we should rename the system folders (https://github.com/devonfw/devon-ide/tree/b9cca7862995eafbac07000f55eafa5d639019c7/scripts/src/main/resources/system) to the according OS - hence Finder should be named or be move to folder MacOs and IDEenv should be named Windows (optionally subfolder Explorer). This is especially helpful when someone downloads the all platforms release containing all the contents for any OS.

We want to redunce the number of scripts in the top-level directory (DEVON_IDE_HOME) to a minimum (see #31). The plan is: there is only setup and its windows variant setup.bat.
However, newbie users and non shell freaks may still be lost in space after calling setup and do not know how to get started (e.g. launch eclipse).
Therefore the idea is that (very few) additional scripts will be generated during setup:

• eclipse-main[.bat] ?
• vscode-main[.bat] ?
• console[.bat] ?

I definitely only want to generate the scripts for the actual platform (so *.bat on windows and bash scripts without extension for linux/mac). However still I want to challenge this:

• Do we need console[.bat] anymore?
• IMHO we should only generate the IDE scripts
  • for the IDEs that are configured for the project (contained in TOOLS variable)
  • for the main workspace by default (if there are other workspaces you would need to create them via CLI e.g. devon eclipse create-script but we want to get the people more used to cli so why not just call devon eclipse instead of double-clicking the generated script. Those users who like to have the script can do it as described)

WDYT?

When we have agreed and implemented this plan, we will delete these console* scripts from the top-level directory.

EOL style is not properly configured and hence bash scripts are build with CRLF on windows resulting in a broken release.

Currently env.sh is changing the current working directory (like invoking cd - I assume).
This is confusing and considered as bug.

User @maihacke changed env.sh to use $(dirname "$0"):
b9cca78#diff-59bbdf4f0007715da1871587ba4ffd3d

I assume this was some kind of bugfix for MacOS. Also this seems more reliable to me as $BASH_SOURCE is proprietary and only works in bash. What is the purpose why this was ever changed to this?

User @vapadwal wants to change env.sh back to $(dirname "$BASH_SOURCE"):
985a51e#diff-59bbdf4f0007715da1871587ba4ffd3d

As I do not like to have git wars, I would kindly ask all participants to clarify the reasons for these changes and find a peaceful solution.

Category: Enhancement

Severity: Low

Description of expected enhancement: The current refactoring is a good opportunity to reduce the a bit too generous use of external environment variables by the IDE scripts a bit. Even when those variables are only defined within the workspace consoles - and everything what is started from there - there is a small risk of unnecessary side effects, when variables of local scope like "MODE" are exposed to outside.

even though it is more a domain of PL and Jenkins to do realeases it would however be nice to script a generic standard release machanism:

• check that there are no local changes (otherwise stash, or fail)
• bump version
• commit
• create tag
• build and deploy release
• open next snapshot version

Should support CI friendly maven as well as manual versioning in POMs. Should define default profile release/deploy (currently we standardized deploy in defonfw itself) for release build.

See also oasp/oasp4j#662 and do the analogue thing for oasp4j-ide to devon-ide.

Shall we update configurator to the latest version of JSON-P?
https://github.com/eclipse-ee4j/jsonp/

Why do things move from java standard groupId (javax.json) to more proprietary groupIds (org.glassfish and jakarta.json)? All political non-sense...

With our commandlet infrastructure it is easy to add support for SonarQube. Allow devon sonar «args» to install, setup, start/stop sonarqube and inspect projects into local sq.

Documentation (specification) is already updated:
https://github.com/devonfw/devon-ide/wiki/configuration

Scripts need to be updated accordingly...

WORKSPACES_PATH is defined here:
https://github.com/devonfw/devon-ide/blob/199ba7ab931e293fd8c8629ef2535b40622d36e7/scripts/src/main/resources/variables#L12
(do not ask me why this is made configurable - we should actually avoid such pointless flexibility - see also #5)
The bug comes here:
https://github.com/devonfw/devon-ide/blob/199ba7ab931e293fd8c8629ef2535b40622d36e7/scripts/src/main/resources/scripts/environment-project#L16
as this results in WORKSPACE_PATH being relative.
However, Eclipse requires the workspace path to be absolute rather than relative and further eclipse might be started from a different CWD.
On Windows the WORKSPACE_PATH is already absolute:
https://github.com/devonfw/devon-ide/blob/199ba7ab931e293fd8c8629ef2535b40622d36e7/scripts/src/main/resources/scripts/environment-project.bat#L15

Therefore linux script needs to be fixed to make the path absolute.

Currently IntelliJ is not yet supported. With our generic infrastructure this is now easy to add.

If you setup your IDE (using source env.sh, using IDEenv, running console, etc.) several environment variables are set/modified (e.g. JAVA_HOME, PATH, etc.). If this happens more than once (maybe even with different parallel IDE project installations) this can get a little messy:
Some environment variables like PATH are not entirely overwritten but extended. So e.g. for PATH with every call several directories are prepended. So if you have multiple of these IDE environment variable setups you will have a very long and messy path.
It would be awesome to find a solution how we can detect this and replace the previous extensions rather than extending again.
To make it very clear, here is what I mean:

➜  devon source env.sh
IDE environment has been initialized.
➜  /projects echo $PATH
/projects/devon/software/maven/bin:/projects/devon/software/java/bin:/projects/devon/software/eclipse::/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/opt/X11/bin
➜  /projects cd mmm
➜  mmm source env.sh
IDE environment has been initialized.
➜  /projects echo $PATH
/projects/mmm/software/maven:/projects/mmm/software/java/bin:/projects/mmm/software/eclipse:/projects/devon/software/maven/bin:/projects/devon/software/java/bin:/projects/devon/software/eclipse::/projects/devon/software/maven/bin:/projects/devon/software/java/bin:/projects/devon/software/eclipse::/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/opt/X11/bin