Sonntag, 22. November 2015

Tangram Release 1.0

Since Tangram reached more than the set of features originally intended back in 2009, it was time to call the current state a 1.0 release.

Why?

The name Tangram points out that the main objective was to create web applications from a fixed set of modules and options which can form nearly any shape you would need. You can start quickly with a limited set of functionality and let the application grow. Fixed set in this case only means that you will need e.g. a model implementation but still have a choice, which one you would want to use. Also the glueing together with a Dependency Injections solution provides several options.
Another intention was to provide reasonable defaults for nearly every aspect where this is possible. You don't need to copy tons of files which do things technically necessary, which you didn't think of at that time. You only set up the things you know that you need them, and all the other features stay quiet. At least they should do something reasonable without getting in your way.

Users

Tangram is in production use since 2011 by Provocon, Ponton, Naturinspiriert.org and others.

Dynamic Extendibility

For many extensions you will not even need a deployment but a change in the data repository forming small and stable cores where the application as whole can be modified to changing requirements quickly, easily and securely.
Apart from the obvious presentation stuff this includes Groovy codes to create and parse URL, extend the business logic, and even the model layer with new model classes.

Core Features

We provide object oriented templating for objects from JPA, JDO, EBean, and CoreMedia repositories. The core system emphasises dynamic web programming through CSS, JavaScript, Velocity Templates, and Groovy Codes in the repository together with basic CDN support, code minification, and caching support.

URLs can be formatted in nice, SEO friendly schemes and be easily mapped to actions to provide the result views for users.
The implementation of authentication and authorization now uses the pac4j set of libraries to provide a seamless and easy integration of plain user lists in files or the repository, OAuth providers, OpenID providers, Google App Engine user service and others for the web application and the base system itself.
This authentication solution is used grant access to the system itself but also to provide support for protected content areas in your applications. You only have to focus on the question, which content needs to be protected by a access granting scheme. Logins can be a generic login page or integrated in you application design.

Generic Editor

Except for CoreMedia we provide a generic web editor which is now responsive (there is now separate mobile editor anymore) which can be loaded from the respective cloud locations of the used components (CKEditor, CodeMirror). Also it can be extended with our dynamic web programming facilities.

Glueing Stuff Together

The mentioned default set of configurations and the option to customize this to your needs is achieved by Dependency Injection libraries. Tangram supports the usage of the Spring Framework, Google Guice, and dinistiq. It should be possible to add other solutions as well provided that they offer a decent set of functionality (which is not always the case for smaller DI libs.) including e.g. optional values, overriding of configurations, and deep generics introspection.

Readware

To illustrate the usage and provide a nice starting point, a set of example applications is provided in sync with the releases.
A documentation wiki is now starting.

Where the 1.0 Release Ends

One real limitation is the older set of JSP/Servlet APIs which need to stay in place to support the Google App Engine. So this is the final release to support the App Engine to be able to move ahead to newer APIs for several areas. This will not be achieved in the 1.0 branch.

Mittwoch, 28. Oktober 2015

Gradle Plugin for EBean, JPA, and JDO Enhancing along with Minification and Overlaying

During the development of the Tangram framework project a set of build related things went into a plugin for the Gradle tool.
The functionality of this Tangram Gradle Plugin is only in very small parts directly related to Tangram. It is more or less a general purpose plugin for applications needing
  • Byte-code transformation of model classes for
    JDO, JPA, and Ebean ORM layers
  • Minification of CSS and JavaScript codes to be placed in WAR artifacts
  • Support underlying of WAR files into others (similar to overlays)
The good news for today is, that with the latest version 1.0.5 the plugin can be used from the central Gradle plugins repository with the - not very surprising - id tangram.gradle.plugin. So some of the usage notes have to be aligned with this situation.

Usage

Just a few lines have to be added to your Gradle build script to use the plugin:

plugins {
  id "tangram.gradle.plugin" version "1.0.5"
}

All of the following steps described here take place without any additional configuration.

Prepare EBean, JPA, and JDO Model Classes

When used with Java projects - and when some data model classes are discovered, - the plugin tries to prepare them for use the respective Object Relational Mapper (ORM). The ORM APIs supported are
These APIs in turn are supported by a number of implementations. The supported implementations are
These OR-Mapper API implementations require (DataNucleus and EBean) or recommend (the others) to apply byte-code transformations called "Enhancing" or "Weaving" to the class files. The compiled code is extended with some database access support to implement the active record pattern more or less seamless.
The API and implementation library in use is discovered from the names of the elements of the class path of the project. If one of the mentioned libraries is found, the corresponding byte-code transformation is applied to the appropriate step (post compile or pre jar creation).

martin@nelson:~/proj/tangram/sites/naturinspiriert$ gradle clean build
:clean
:compileJava
Performing DataNucleus JPA byte code transformation.
ENHANCED (Persistable) : org.naturinspiriert.RootGroup
ENHANCED (Persistable) : org.naturinspiriert.Topic
ENHANCED (Persistable) : org.naturinspiriert.AbstractGroup
ENHANCED (Persistable) : org.naturinspiriert.ImageData
ENHANCED (Persistable) : org.naturinspiriert.Article
ENHANCED (Persistable) : org.naturinspiriert.Linkable
ENHANCED (Persistable) : org.naturinspiriert.Group
DataNucleus Enhancer completed with success for 7 classes. Timings : input=60 ms, enhance=54 ms, total=114 ms. Consult the log for full details
7 classes enhanced.
:processResources
:classes
:jar
:war

:assemble
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:check UP-TO-DATE
:build

BUILD SUCCESSFUL

Total time: 4.53 secs

(Highlighted message to indicate use of the DataNucleus Enhancer)

The byte-code transformations directly use the transformer of the respective library in use except for the OpenJPA case, where the ant task of the enhancer is integrated. Some of the transformers issue some logging.

Switching Off

Additionally it is possible to switch of the byte code transformation for JPA annotated classes by adding to your build file

// build.gradle: 
enhancer.enabled=false

in case this might be necessary e.g. to only use the other parts of the plugin. Also EclipseLink, Hibernate, and OpenJPA support the use of plain Java classes without the byte-code transformation.

JavaScript and CSS Minification

When used in conjunction with the war plugin, CSS and JavaScript resources are automatically minified.
The plugin checks for resources with a filename extension .css for Cascading Style Sheets and .js for JavaScript. Matching resources are minified using the YUI Compressor.
It is not possible to minify resource included from archive files but only file resources local to your project. So contents from archives - while being included in the resulting web archive - cannot be minified. (We would expect WAR files to contain minified resources like WAR files generated using this plugin do.)

Web Application Underlying

The plugin introduces a configuration named webapp for modules using the war plugin. Dependencies given for this configuration are extracted into the resulting war artifact.

// build.gradle:
dependencies {
  webapp "tangram:tangram-core:$tangram_version:war@war"
  webapp "tangram:tangram-editor:$tangram_version:war@war"


  compile "tangram:tangram-core:$tangram_version"
  // Persistence API JPA
  compile "tangram:tangram-jpa:$tangram_version:nucleus"
  compile "org.datanucleus:datanucleus-api-jpa:$versions.datanucleus"
  compile "org.datanucleus:datanucleus-core:$versions.datanucleus"
  compile "$versions.jdo_api"
  compile "$versions.persistence_api"
  runtime "org.datanucleus:datanucleus-mongodb:$versions.datanucleus"

  compile "tangram:tangram-editor:$tangram_version"
  runtime "tangram:tangram-dinistiq:$tangram_version"
  runtime "org.slf4j:slf4j-log4j12:$versions.slf4j"


  providedCompile "$versions.servlet_api"
  providedCompile "$versions.jsp_api"
}


This process is not really described well if called overlay so I call in underlying, since your web application's directory in fact is the overlay so the other archives referenced and included must be an underlying.
If your WAR relies on the contents of another pre-packaged or incomplete WAR, the contents of the latter will be copied into your resulting web application while you can override any file in this archive from your local web application contents directory.

Version List

The plugin introduces a version object which collects some version strings for a number of libraries. This ensures that any project using the plugin can use these libraries with recent versions and version changes are applied in sync. The use of this part is optional and you have to explicitly use the versions in your build file since this cannot be applied transparently.
Some random examples:

dependencies {
  compile "org.pac4j:pac4j-openid:$versions.pac4j"

  compile ("org.apache.openjpa:openjpa:$versions.openjpa") {
    exclude group: 'asm'
  }

  compile "org.eclipse.persistence:org.eclipse.persistence.jpa:$versions.eclipselink"
  compile "$versions.persistence_api"

  compile "org.hibernate:hibernate-core:$versions.hibernate"
   
  compile "org.datanucleus:datanucleus-api-jpa:$versions.datanucleus"
  compile "org.dataucleus:datanucleus-core:$versions.datanucleus"
  compile "$versions.jdo_api"
  runtime "org.slf4j:slf4j-log4j12:$versions.slf4j"

  testCompile "org.testng:testng:$versions.testng"

  // your container will have this for you
  providedCompile "$versions.servlet_api"
  providedCompile "$versions.jsp_api"
}


Manual Mode

Of course it is still possible to call the methods performing the different tasks directly like described in the 0.9 plugin blog post. This should only be necessary if you e.g. want to enhance files in the unit test section of your code, which is considered a very rare case.

Dienstag, 27. Oktober 2015

Dinistiq Version 0.5

The minimalistic Dependency Injection library for Java - Dinistiq - has been published in version 0.5.
This version still doesn't change the main outline of the project: It deals with a single - singleton - scope with a few extensions in a very small footprint library. The library does not introduce any custom tags but fully relies on a subset of the JSR330 definitions.
Despite its limited scope it still successfully executes the TCK and comes with a reasonable unit test coverage. These unit tests have been migrated to TestNG to be in sync with my other projects reflecting the license which better fits into the set of licenses of the used dependencies in this project.

Fixes

The once more extended test coverage after the migration to TestNG showed two bugs relating to class discovery. These have been fixed in this release.

Leaner Web Integration

Due to the use of older versions of the Servlet API, dinistiq had to use a central dispatcher servlet dispatching to other so called registerable servlets. This level of indirection has been complete removed. With dinistiq 0.5 the web integration is mostly rewritten now depending on the Servlet API version 3.1.
Sadly this also removes its URL pattern matching capabilities.
RegisterableServlet instances from the dinistiq scope are now registered with the Servlet container directly. They now present a number of Servlet specification compatible URL patterns (no regular expressions any more - sorry) and and the same ordering indicator as with previous releases.

Availability

Starting from release 0.4 dinistiq is available from the JCenter repository at

https://jcenter.bintray.com/dinistiq/

Latest snapshots are provided at

https://oss.jfrog.org/oss-snapshot-local/ 

There are already some 0.6-SNAPSHOTs available, but they don't present changes worth mentioning right now.

Migration

Take a look at your web.xml. If you see something like

  <servlet>
    <servlet-name>dinistiq</servlet-name>
    <servlet-class>dinistiq.web.DinistiqServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>dinistiq</servlet-name>
    <url-pattern>/s/*</url-pattern>
  </servlet-mapping>


simply remove it!
The RegisterableServlet interface has been change as mentioned. You will have to update you regular expressions to Servlet specification compatible URL patterns. Of your the respective method signature was changed from

    @Override
    public Set<String> getUriRegex() {
        Set<String> result = new HashSet<>();
        result.add("");
        result.add("/.*");
        return result;
    } // getUriRegex()

to

    @Override
    public Set<String> getUrlPatterns() {
        Set<String> result = new HashSet<>();
        result.add(source.getDispatcherPath()+"/");

        result.add(source.getDispatcherPath()+"/*");
        return result;
    } // getUrlPatterns()

Mind the missing regular expression syntax and that the full URI path has to be given.


Samstag, 3. Oktober 2015

Gradle Plugin for JDO, JPA, Ebean etc in Tangram grown up

As part of the Tangram framework project a set of build related things went into a plugin for the Gradle build tool.
Only in part this plugin is directly related to Tangram but offers some generic tools suitable for many projects.
The plugin does
  • Support underlying of WAR files into others (similar to overlays)
  • Minify CSS and JavaScript codes to be placed in WAR artifacts
  • Byte-code transform model classes for JDO, JPA, and Ebean ORM layers
Up to version 0.9 the plugin was more or less a collection of utility methods. It is now redesigned to do the same job transparently when appropriate and at the appropriate stage of the build process.

Usage

Just a few lines have to be added to your Gradle build script to use these add on functionality:

buildscript {
  repositories {
    jcenter()
  }
  dependencies {
    classpath 'tangram:gradle-plugin:1.0.0'
  }
}


apply plugin: 'tangram'

All of the following steps described here take place without any additional configuration.

Web application underlying

The plugin introduces a configuration named webapp for modules using the war plugin. Dependencies given for this configuration are extracted into the resulting war artifact.
This process is not really described well if called overlay so I call in underlying, since your web application's directory in fact is the overlay so the other archives referenced and included must be an underlying.
If your WAR relies on the contents of another pre-packaged or incomplete WAR, the contents of the latter will be copied into your resulting web application while you can override any file in this archive from your local web application contents directory.

Minification

When - again - used in a project also using the war plugin, all files included from the project directory will be checked for CSS and JavaScript file extensions. Matching files are minified using the YUI Compressor.
It is not possible to minify resource included from archive files. So contents from archives while being included in the resulting WAR cannot be minified. (We would expect WAR files to contain minified resources like WAR files generated using this plugin do.)

Handle Model Classes

When used with Java projects and when some data model classes are discovered, the plugin tries to prepare them for use with one of the ORM frameworks Java Data Objects (JDO), Java Persistence API (JPA), or Ebean.
The OR-Mappers require or recommend to apply byte-code transformations called "Enhancing" or "Weaving" to the class files for the data model related classes.
The plugin supports DataNucleus Access Platform (JDO and JPA), EBean, EclipseLink, Hibernate, and OpenJPA. All the calling options from the 0.9 version of the plugin have be removed and the APIs are auto-discovered so the whole process is now - except for logging output - transparent during build.

Version List

The plugin introduces a version object which collects some version strings for a number of libraries. This ensures that any project using the plugin can use these libraries with recent versions and version changes are applied in sync. The use of this part is optional and you have to explicitly use the versions in your build file since this cannot be applied transparently.
Some random examples:
dependencies {
  compile "org.pac4j:pac4j-openid:$versions.pac4j"

  compile ("org.apache.openjpa:openjpa:$versions.openjpa") {
    exclude group: 'asm'
  }

  compile "org.eclipse.persistence:org.eclipse.persistence.jpa:$versions.eclipselink"
  compile "$versions.persistence_api"

  compile "org.hibernate:hibernate-core:$versions.hibernate"
   
  compile "org.datanucleus:datanucleus-api-jpa:$versions.datanucleus"
  compile "org.dataucleus:datanucleus-core:$versions.datanucleus"
  compile "$versions.jdo_api"
  runtime "org.slf4j:slf4j-log4j12:$versions.slf4j"

  testCompile "junit:junit:$versions.junit"

  // your container will have this for you
  providedCompile "$versions.servlet_api"
  providedCompile "$versions.jsp_api"
}

Manual Mode

Of course it is still possible to call the methods performing the different tasks directly like described in the 0.9 plugin blog post.
Also it is possible to switch of the byte code transformation for JPA annotated classes by adding to your build file

// build.gradle: 
enhancer.enabled=false

in case this might be necessary e.g. to only use the other parts of the plugin.

Sonntag, 26. April 2015

Stock Apache Tomcat and Gradle on OpenShift

For my ongoing work on Tangram a wanted to use the RedHat OpenShift platform. In this post I'm going to outline the generic part of this which now is extracted as a Quickstart on github via https://github.com/mgoellnitz/openshift-tomcat-gradle-quickstart.
The requirements where the following: Build must be possible with Gradle and a plain Apache Tomcat webcontainer - not a JBoss Tomcat derivative - is needed. For the time being I still left out the option to use Jenkins, since the way to re-deploy on git-push OpenShift gives me, is quite usable.
So this results in the idea of a combination of the work of Shekhar Gulati at https://www.openshift.com/blogs/run-gradle-builds-on-openshift with the code at https://github.com/shekhargulati/gradle-openshift-quickstart and the tomcat Quickstart https://github.com/openshift-quickstart/openshift-tomcat-quickstart
Looking at the changes needed to use Gradle, I decided to integrate them manually and modify them along the way. Also the Tomcat Quickstart will not be used directly but serves as a template, since I don't want to pull the Apache Tomcat binaries into the git repository. As opposed to many other examples I came across, the installation of the RedHat cloud command line tools is completely left out. You might want to use them, but they are not needed for this example.
First of all I had to create an application (named "test") and had to take "DIY 0.1" as the type of application and clone the resulting repository.

git clone ssh://524e65...094@test-application.rhcloud.com/~/git/test.git/
cd test


This repository resembles the working directories of your application on OpenShift and is not just a development set of folders. So you will find some scripts here to interact with the infrastructure of OpenShift. These scripts have to be customized to fetch and configure Apache Tomcat and Gradle.
Now it's time to add script element to fetch the tool.

# .openshift/action_hooks/build:

# gradle build
set -x

DIR=`pwd`

# obtain tomcat
export TOMCAT_VERSION=7.0.61
if [ ! -f $OPENSHIFT_DATA_DIR/apache-tomcat-${TOMCAT_VERSION}.tar.gz ] ; then
  wget http://apache.../v${TOMCAT_VERSION}/bin/apache-tomcat-${TOMCAT_VERSION}.tar.gz
fi
if [ -f $OPENSHIFT_DATA_DIR/apache-tomcat-${TOMCAT_VERSION}.tar.gz ] ; then
  rm -rf tomcat
  tar xvzf $OPENSHIFT_DATA_DIR/apache-tomcat-${TOMCAT_VERSION}.tar.gz
  mv apache-tomcat-${TOMCAT_VERSION} tomcat
  cd $OPENSHIFT_DATA_DIR/tomcat
  rm -rf logs
  ln -s $OPENSHIFT_LOG_DIR logs
  cd $OPENSHIFT_DATA_DIR
fi

# obtain gradle
export GRADLE_VERSION=2.2.1
if [ ! -d $OPENSHIFT_DATA_DIR/gradle-${GRADLE_VERSION} ] ; then
  cd $OPENSHIFT_DATA_DIR
  wget http://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-bin.zip
  unzip gradle-${GRADLE_VERSION}-bin.zip
  rm -f gradle-${GRADLE_VERSION}-bin.zip
fi

cd $DIR

cd $OPENSHIFT_REPO_DIR
export GRADLE_USER_HOME=${OPENSHIFT_DATA_DIR}gradle
export GRADLE_HOME=${OPENSHIFT_DATA_DIR}gradle-${GRADLE_VERSION}
export PATH=$GRADLE_HOME/bin:$PATH
if [ -f openshift-build.gradle ] ; then
 gradle --stacktrace -b build.gradle clean build
fi


Use at least version 2.2 of Gradle due to http://issues.gradle.org/browse/GRADLE-2871. Also Gradle needs some local directory on the target host.


# .openshift/action_hooks/pre_build:
 

# gradle storage dir build
set -x

if [ ! -d $OPENSHIFT_DATA_DIR/gradle ] ; then
  mkdir $OPENSHIFT_DATA_DIR/gradle
fi


The default stop action hook script contains some references on a ruby server and is missing a nice tomcat handling.


# .openshift/action_hooks/stop:
source $OPENSHIFT_CARTRIDGE_SDK_BASH

# The logic to stop your application should be put in this script.
if [ -z "$(ps -efww | grep tomcat | grep -v grep)" ] ; then
  client_result "Tomcat is already stopped"
else
  if [ -x $OPENSHIFT_DATA_DIR/tomcat/bin/shutdown.sh ] ; then
    $OPENSHIFT_DATA_DIR/tomcat/bin/shutdown.sh
  else
    kill `ps -efww | grep tomcat | grep -v grep | awk '{ print $2 }'` > /dev/null 2>&1
  fi
fi
exit 0



For safety reasons it might be a good idea to delete unexploded wars from the webapp directory of the tomcat installation


# .openshift/action_hooks/deploy:
set -x

# Addition:
for i in `ls $OPENSHIFT_DATA_DIR/tomcat/webapps/*.war`; do
  dn=`basename $i .war`
  echo "removing $dn"
  rm -rf $OPENSHIFT_DATA_DIR/tomcat/webapps/$dn
done


The start script has to be modified not to use ruby but the tomcat installation


# .openshift/action_hooks/start:
set -x
cp -rdp ${OPENSHIFT_REPO_DIR}/tomcat/conf/* ${OPENSHIFT_DATA_DIR}/tomcat/conf/
cp -rdp ${OPENSHIFT_REPO_DIR}/tomcat/lib/* ${OPENSHIFT_DATA_DIR}/tomcat/lib/
cp -rdp ${OPENSHIFT_REPO_DIR}/tomcat/webapps/* ${OPENSHIFT_DATA_DIR}/tomcat/webapps/
cd $OPENSHIFT_DATA_DIR/tomcat
sed -ig 's/OPENSHIFT_DIY_IP/'$OPENSHIFT_DIY_IP'/g' conf/server.xml
sed -ig 's/OPENSHIFT_APP_DNS/'$OPENSHIFT_APP_DNS'/' conf/server.xml
bin/startup.sh


After these preparation steps it would be time to integrate your application by providing a build.gradle script in the root path and all the needed source files.
When using a custom domain, let this domain point to your Applications hostname via a DNS CNAME record. Then create an alias with your domain name for the application using the RedHat Cloud command line tool. (Or create the alias via the console web application) Don't forget to add this domain name to your server.xml override.

Mittwoch, 25. März 2015

Dinistiq Version 0.4

The minimalistic Dependency Injection library for Java Dinistiq has been published in version 0.4.
This version doesn't present anything specifically new. It still deals with a single - singleton - scope with a few extensions in a very small footprint library. The library does not introduce any custom tags but fully relies on an subset of the JSR330 definitions.
Despite its limited scope it still successfully executes the TCK and comes with a once more extended unit test coverage.

Availability Release

One of the main changes of this release is the availability of the release artifacts through the JCenter service of bintray. This renders the integration of Dinistiq even easier since you will most likely not need to add another repository or mirror to your infrastructure.
Dinistiq is in daily production use for some time now and will remain supported as long as there is not always a need for more feature complete solutions like Google Guice or the Springframework.

Next Steps

The versioneye service shows that Dinistiq is up to date with its dependencies with only one intentional exception. This release will be the last to support the Google App Engine through the use of outdated Servlet API versions.