oasp / oasp4j Goto Github PK

currently the JAX-WS exception facade is creating a custom string with error information in case an error occurred. Format is e.g. {UUID}:{message}. Instead we should use JSON for this purpose so the client can rely on the payload and does not have to implement a strange parser (sometimes UUID is present, sometimes not, etc.).

{ "message": "my example error message",
"uuid": "eff89....",
"code": "ORD42" }

We have created a minimal infrastructure for monitoring and related aspects such as loadbalancer status.
We need to consolidate the best practices we have for JMX and integration into tools such as nagios, etc.
This should result in improvements to our monitoring module and the sample application.

We created the oasp4j-documentation module to build the documentation from our wiki as PDF or epub. This is currently only a PoC.
We need to solve the following issues:

• Internal links and anchors shall work in gollum that has limited asciidoc support. We also want them to work in the generated documentation.
• Wiki pages shall contain a TOC as this is helpful. However, the generated documentation including these pages shall only have a single TOC for the whole thing.
  In order to appear in the TOC the sections have to be level 1, 2 or 3. However, we need the right level of sections for including in the overall documentation.

Our current idea is to write a simple pre-processor (e.g. via maven-antrun-plugin) that resolves all these issues by generating transformed asciidoc out of the original ascidoc pulled from the gollum wiki.

So for foo.asciidoc
= Page Heading
...
xref:page-heading[top]
link:sometopic#some-heading[link]

the pre-processor would produce this transformed asciidoc:
=== Page Heading
[[foo_page-heading]]
...
xref:foo_page-heading[top]
xref:sometopic_some-heading[link]

We have created a generic exception facade for JAX-RS in our sample application. We should move this code to a new oasp4j-service module (maybe seperate between WS and RS).
For JAX-WS we should try to implement the same thing and integrate it properly with the JAX-WS framework implementation. See also issue #29.

For our IDE settings we need some rework. Some templates for JavaDoc, etc. are in German. Also we should avoid redundant and overloaded information.

/**

• This method returns the last name.
• @return the last name. May be null if not set.
  */

Instead this is already sufficient:

/**

• @return the last name. May be null if not set.
  */

New types currently get $Id$ generated as version what is pointless and shall be removed (in template and existing code).

We need to verify that checkstyle is properly preconfigured and also the dictionary.

Create an (internal) release 0.2.0, approve, distribute, and announce it.
We are leaving alpha state and enter beta with this version.

I just realized that the configured integration class for plugging Jackson into CXF is provided by Jackson itself:

<jaxrs:providers>
<bean class="org.codehaus.jackson.jaxrs.JacksonJsonProvider">

We should therefore start an evaluation if we are ready to upgrade to the new version (com.fasterxml.jackson) as for the long run org.codehause.jackson is a dead end.

For transfer objects we are currently using business objects (with suffix BO) that are a simple and plain equivalent of the according entities. However with such approach you easily run into the problem that you have no clear concept which "selection" of your data model you want to transfer.
Instead there is a commonly used pattern that we call the DTO and CTO pattern.
You can find a public summary of the concept here:
http://m-m-m.sourceforge.net/apidocs/net/sf/mmm/util/transferobject/api/package-summary.html

While the current BO approach is KISS as it is simple, it is not solid as you easily run into problems and have no clear guidance. If you need something more complex anyways sooner or later, then for KISS it is still better to always follow a single and solid approach than saying in case A do x1 and in case B do x2.

If we bundle up the sample app of oasp4j together with the oasp4js sample things need to work together properly. In beans-security.xml we defined this config line:

Is the plan that the client is located in oasp4j-example-application/jsclient/* ?
So in order to start the application one needs to open the pathoasp4j-example-application/jsclient/index.html?

Shouldnt the index.html be located directly in the toplevel folder of the web application and therefore also added to beans-security.xml?
Should we not just use something like "static" or "resources" instead of "jsclient" in order to have a generic path schema for general purpose? So /static/* would imply that everything is static content here?
BTW what about the plans to have server-side logic for the HTML templates of the AngularJS dialogs? Such as I18N and Security...? At least then static would not fit properly and the development of the oasp4js app would be even more dependent on the backend. However, this approach still makes a lot of sense to me...

Feedback is most welcome.

We need to provide a template for an OASP4J application. Technically this could be realized as maven archtype. We should wait until the sample application has matured and we can easily extract it from there by removing the individual code.

We want to be able to easyly test our services with junit. The goal is to start a minimal server to test different services within the junit test.

We have created a really cool documentation infrastructure and also want to use if for other things such as OASP4JS or some of our tools.

JSON payload format of rest services changed accidentally (due to issue oasp/oasp4j-sample#14).
Now we get this:
["java.util.ArrayList",[{"@type":"TableBo","id":1,"version":0,"waiterId":null,"state":"OCCUPIED"},{"@type":"TableBo","id":2,"version":0,"waiterId":null,"state":"FREE"},{"@type":"TableBo","id":3,"version":0,"waiterId":null,"state":"FREE"},{"@type":"TableBo","id":4,"version":0,"waiterId":null,"state":"FREE"},{"@type":"TableBo","id":5,"version":0,"waiterId":null,"state":"FREE"}]]

"java.util.Arraylist" has nothing to do here as well as the extra array wrapping.
Also @type should only occur for polymorphic types and with symbolic names but not "TableBo".

Currently we have no strict schema for our service URLs. We have REST services that are all lowercase and we have SOAP services using CamlCase. Also our documentation is not yet complete for this aspect (CamlCase, versioning, etc.).
While I personally like CamlCase most REST guys consider this as bad practice:
http://stackoverflow.com/questions/778203/are-there-any-naming-convention-guidelines-for-rest-apis
http://ux.stackexchange.com/questions/9303/underscores-vs-hyphens-in-urls

An option would be to use CamlCase but treat the path as case insensitive but this is not the default as URL paths are actually case-sensitive (even though many people say that URLs are case-insensitve what is actually wrong).
http://stackoverflow.com/questions/5624481/case-insensitive-urls-with-jax-rs
But if we end up with things like @Path( "/{message : (?i)message}" ) then I am for simple lower-case convention and where necessary use hyphens. Currently we do not use hyphens and just have /services/rest/offermanagement/product/.... I think this is OK but OfferManagement or offer-management would be more human readable instead. If CamlCase is not an option hyphens cause extra characters and more typing so should we always use them to separate words or never use them, or what? What we currently have is KISS and it still appears readable to me. So I would suggest for lowercase and avoid hyphens (as terms should be short and easy).

Also for versioning there is a review note in our documentation:

Should we think of versioning individual services? Or are we versioning the entire SOAP or REST API? Would one end up with a mixture of FooService in V2 and BarService in V3? Should we distinguish between major and minor version here (this is more implementation specific and can cause mystic discussions where linear versioning v1, v2, v3 would be more KISS)? For REST services we would add the version as a "folder" of the URL path. Should we do the same for SOAP? What are the implantations to namespacing, etc?

We need a service framework. It should ideally support both JAX-WS and JAX-RS and integrate with CDI and spring.
I managed to do this with CXF in our example (not committed).
Simon says this also works with metro from JDK OOTB. In that case this would be preferred.

If we use custom datatypes they are no regular java beans. Hence we need to figure out how to map them to and from JSON with jackson. In our sample application we already have MonetaryAmount that is currently not mapped correctly. Here we should demonstrate how to do it properly and also describe this in the documentation (guide-datatype and esp. guide-service-layer).

In addition to our documentation we are planning to have a step by step tutorial. This will partially overlap with our existing documentation that is still the heart of all...

Current plan:

1. Use spring-security for authorization and Java standard mechanisms for authentication (JAAS, ServletContainer)
2. Create an integration with Open-Source access manager (JONAS?) based on 1. for example app (restaurant).

we need some configuration module. I already created it as empty with just a constant with the spring standardized configs for reference in unit tests. The config module should at least address:

1. Access configurations (from properties files)
2. Read and Write single attributes
3. Support for reloadable configurations

Maybe something to look at:
http://www.morgan-design.com/2012/08/reloadable-application-properties-with.html

We need a jenkins and preferable also a sonarqube instance. @sobkowiak you talked about a free solution that offers 3 "gears" where we could install something.
Could you try a PoC with jenkins and we will have a look? With cloudbees the free service is too limited. For m-m-m I have a private PC for this purpose what is not suitable for oasp. If we can not use a free service we can either make it internal and host it on a dedicated cg machine or we buy some external service.

The Nightly Build job should deploy the artifacts into Nexus repository (until we can deploy it OSSRH).

Our repository oasp4j is wrongly displayed as repo for XSLT.
https://help.github.com/articles/my-repository-is-marked-as-the-wrong-language

It would be great if someone could fix this but it is very low priority.

Database migration for release upgrades is a big and important topic.
Flyway (http://flywaydb.org/) seems to be a great solution for this problem. I got input and demands for it from various different sources. We should take a close look and consider this for our example...

Initially I did not like the idea of triggering a database migration as "side-effect" when starting the new release as server for the first time. Assume this is triggered by accident in production.
But after rethinking it is more a less a typically fear of something new. You just have to understand something and get used to it. Would operators prefer to have a proprietary shell script from us?
Feedback?

I am convinced that it is perfect for test systems so we should think about it...

I know that it is a total pain but the domain oasp.org was taken while our ordering process was about to buy it. We will keep the name OASP but we can not keep the namespace oasp.org.

Secrity is castring UserProfile to String causing ClassCastException and calling toString on null causing NPE.

Within the logging output as well as e.g. for the purpose of performance monitoring certain context information should be made available. One example is a "correlationId", which can be used to bring together all log entries which belong to a certain instance of a use-case.

The file
https://github.com/oasp/oasp4j/blob/develop/oasp4j-logging/src/main/java/org/oasp/module/logging/common/api/LoggingConstants.java already defines the keys/names for such context information. (At the moment only "correlationId" is defined here.)

In case of logging through the SLF4J-Api these context information needs to be stored in the Mapped Diagnostic Context (http://www.slf4j.org/api/org/slf4j/MDC.html). Even though this provides already an abstraction from the underlying logging implementation (Log4j, Logback, java.util.logging) there might be other components which also require such context information. One example is the performance monitoring framework "Kieker" (http://kieker-monitoring.net/) which uses different structures for holding this context information.

To decouple the setting of these context information from the usage of the information I propose to introduce a central spring bean. This spring bean should implement a very simple ("write-only") interface, only allowing to set context information like in a map:

interface DiagnosticContextFacade {
   public void put(String key, String val) 
}

All code in OASP which will set such context information should do this via this facade.
The implementation bean of this facade might then either only set this information within the SLF4J MDC or might additionaly store this information in any other way as needed by other components.

Any comments, hints or suggestions on this?

Oliver

We need to complete the validation documentation.
I just created an inital draft:
https://github.com/oasp/oasp4j/wiki/guide-validation

See also oasp/oasp4j-sample#16

For bean mapping we should create a very small shim as module. We should also discuss if dozer is the right choice. Alternatives are mapstruct or orika. Dozer might be established but it has proven to be buggy.

According to issue #1 we decided for CXF as JAX-RS implementation. CXF also implements JAX-WS so it would be an option to use it for both JAX-RS and JAX-WS. However we have the greater experience with Metro2 what is also the reference implementation. We finally have to make a decision here.

For the OASP4J we use spring-boot-dependencies as basis. However we add some additional dependency versions and might also override some things some day.
Currently we have this in our oasp4j parent POM that also contains the build logic.
I want to separate this into a separate POM so it can be used by projects as parent POM without interfering the build in any way (potentially also as dependency with import scope - not tested).

should be easy as jenkins supports this OOTB for git:
http://stackoverflow.com/questions/15344955/hudson-jenkins-git-build-all-branches
We still need to evaluate this and e.g. see what happens if you build the job manually.

While the RI of JSR 354 (moneta) is licensed ASL2.0 the API has a proprietary license from Credit Suisse that is not compatible for us (too many restrictions, only evaluation permitted).
This will be solved when the JSR is completed and finalized. As there is no final deadline we do not want to block our own readmap and have to remove it (temporary) from our codebase.
I know that this is a pain as JSR 354 has a quite stable and nice API and solves exactly our problems but we have to face the facts here.

For the persistence infrastructure we have to clarify how to proceed. I added m-m-m in a minimalistic way. I can do a lot more like integration of envers and support for revisioned entities and DAOs.
Simon suggested to remove m-m-m and add reimplement this as OASP module.
Marek is fan of spring-data, while Simon and I dislike the magic aspects of it.

We need own Nexus instance to hos our SNAPSHOTs until we can deploy or into OSSRH. We will need it also to deploy the private modules.

issue:
unable to build example application using both maven from command line ("maven clean install") or in eclipse with m2e plugin

root cause:
1: oasp4j-example-application/pom.xml is referencing non-resolvable parent pom

2: oasp4j-sample/pom.xml is referencing non-resolvable parent pom

org.oasp.java
oasp4j
1.0.0-SNAPSHOT
../oasp4j/pom.xml

expected fix:
from my understanding
1: oasp4j-example-application/pom.xml can not reference oasp4j-sample/pom.xml, the latter is nox existing as it can not be build because of error 2 -> solving the issue stopping oasp4j-sample/pom.xml from being build should also fix this problem

2: oasp4j-sample/pom.xml can not reference oasp4j/pom.xml as oasp4j/pom.xml is defined to be adressable with the following maven coordinates:

org.oasp.java
oasp4j-dependencies
1.0.0-SNAPSHOT

so referencing this pom using coordinates

<groupId>org.oasp.java</groupId>
<artifactId>oasp4j</artifactId>
<version>1.0.0-SNAPSHOT</version>
<relativePath>../oasp4j/pom.xml</relativePath>

can not work because of wrong artifactId. it should be using, oasp4j-dependencies right?!?!

as this is a fundamental change in the build setup i am reluctant from simply fixing and checking in my suggested fix and ask for some educated advice - is my assumption wrong or do i follow the wrong steps in building the application? from the given state checked in ci and all other users seem to be able to build the application, but i simply dont get it where i am wrong and dont want to break the running build. or maybe there is something in the maven pom tree submodules hirarchies that i am not able to see or understand how it works

We use optimistic locking via the @Version Long attribute.
However, hibernate updates the version when the transaction gets committed. This is done after the entities are converted to transfer-objects so as a result of a write operation the client will receive wrong versions.
The two possible approaches are:

• use an explicit flush() on entityManager
• keep an internal transient reference of the mapped entity in the DTO. The method getVersion() (or getModificationCounter(), however you call it) implicitly updates the version from the internal reference if not null. After that TX is closed and before serialization the version has to be updated e.g. by recursively calling the getter.

I invented the last concept in one of my projects and we used it with great success. I think it is better than using flush() as this is easily mistaken and can cause performance issues in complex/cascaded calls.

We need to address the following open points for our documentation:

1. target audience
2. application vs. enterprise scope
3. instructions vs. explanations
4. writers style

For 1. we want to strongly focus on developers. However, also designers and architects need answers. Currently we do not want to have multiple guides for different audience. If there is additional information for others than developers we keep them at the end of each page to address developers first with the important aspects.

For 2. we need a solution. In general we want to follow KISS and should therefore address mid-size projects that "only" want to implement an application. We could have a standard section at the end of each guide for aspects that differ in enterprise landscapes (e.g. SSO auth in central portal server instead of single app, central service gateway with SOAP and REST services without Tx, etc.)

For 3. we want to focus on giving guidance in form of something like a tutorial. Explanations (beyond a single sentence as motivation) should be treated like 1. and go to the end.

For 4. we should use use short, personal and straight instructions:
"We use ...", "We always do ...", "We never do ...", "We avoid ...".

But we should not say "The OASP recommends to ...", "It is better to ...", "You must ...".

Lombok is a project which reduces amount of boilerplate code written in every java project. Autogenerated getters, setters, equals, hashCode functions and even loggers.
Maybe using Framework which allows to generate this code for us would be reasonable. Colleague from other project used it in big commercial project with grate success. What do you think about it ?

The CXF community has released some weeks ago a new fix release with over 80 fixes. We should upgrade to the new version

The architecture documentation has to be written. We need updated diagrams, etc.
https://github.com/oasp/oasp4j/wiki/architecture

IMHO we should address both application architecture and enterprise architecture.

https://github.com/reinert/JJSchema provides annotation based json schema generator. Which then can be used by jackson based parser to validate the response. Initial impression is good , one of the attributes available is pattern , which allows for Regex and helps in taking care of JSON fuzzers.

The documentation of the logic(al) layer needs a complete rework:
https://github.com/oasp/oasp4j/wiki/guide-logical-layer

We have created a PoC how to map a composite datatype (MonetaryAmount) via hibernates composite usertype. This is working for the mapping but causing problems in queries:
You can not create queries with selection based on the individual attributes of the composite datatype.
IMHO this is a bug or at least a general deficit in hibernate. Why does it work with embedabbles? Because they are part of the JPA spec and UserTypes are an hibernate proprietary feature?
https://forum.hibernate.org/viewtopic.php?f=1&t=989966

We should therefore change our solution. Even if we currently have no problems as we do not need such queries (yet) it is a potential problem. We do not want to have a sample that is containing misleading patterns that will guide projects on the wrong track!

The only solution is IMHO a workaround. Possibilities are:

• avoid the datatype at all in your entitites and perform according conversions in the bean-mapper.
• create an equivalent java bean for the datatype in the persistence layer and declare it as @Embedabdle. However, the mapping from the datatype to the embeddable bean will work but vice-versa it will still require an according conversion in the bean-mapper.

the service documentation is already quite helpful but for SOAP/WS essential parts are missing and we need to add details examples for code and config. From the sample application we are already far enough to provide the fitting examples...

The guide persistence needs to be updated as description does not always match the shown examples.

For message queuing we will use the JMS standard. However, the technical decisions have to be clarified. Also we have to think about a natural example how to integrate this into the restaurant (maybe as an addon if this would make the example too complex or more complicated to start from the IDE but maybe it can be integrated smooth and easy).
For ActiveMQ Simon can tell a long story about the trouble they had in production and finally they had to replace it completely canceling their entire roadmap.

We currently have this configuration layout

• src/main/resources
  • config
  • resources

Where config contains configurations with the aspects

• under the control of operations
• containing properties with hostnames, passwords, ports, timeouts, etc.
• typically environment specific operations
• typically changes independent from software releases/deployments

While resources has the aspects

• under the control of developers
• containing spring XML configs, etc. with classnames, etc.
• independent from environments
• changes with software releases/deployments and should never be touched from outside

The current layout is not intuitive because

• both are configurations (config) and both are resources
• the naming of the folders is therefore not intuitive
• resources is causing confusion with the resources from maven (duplication)

Instead we should have some more intuitive layout with still a clear separation of these two.

as eclipse project configuration is quite buggy together with the (even buggy) m2 maven plugin it can come to tricky situations in getting the project facet settings in eclipse right. fixing errors wil start m2e to reneter wrong values. this is quite annoying for anyone not known to the details of eclipse facets (always use apply for every step).

as i have been told only source, resource and pom.xml files should be checked into repository. also versioning .classpath and .project introduces other problems, but less then fiddling with tricky eclipse configuration.

Currently images are not working in the generated documentation.
I have already added a build step in the ant script that copies them to the target/docbook folder but it still does not work. I need to do some more investigation what is going wrong...

I see that we are using Dozer for object-object transformation , which is a common aspect in data driven application. Is Dozer optimal and performant , i used it long time back and it was too slow , is it better now ?
Do we have some guidelines where to use it and where to rely on Manual copy.

Not sure the reliability but seems some comparision have been done http://blog.sokolenko.me/2013/05/dozer-vs-orika-vs-manual.html