Git Product home page Git Product logo

grails-ignite's Introduction

Grails Ignite Plugin

A plugin that provides basic Grails integration with the Apache Ignite compute grid framework.

Branch structure

  • master compatible with Grails 2.x
  • grails3 compatible with Grails 3.x
  • grails4 compatible with Grails 4.0.x

Supported Features

Under Development

  • Clustered Hibernate L2 Caching

Grid Bean

The plugin provides a configured instance of the Ingite grid as a bean called "grid", which you can access via injection in controllers and services:

def grid

Configuration

In order to support hibernate l2 caching, which requires the Ignite grid to be started prior to the sessionFactory and therefore the vast majority of Grails artifacts, Ignite must be configured from external configuration files.

The external files must be referenced in the ignite configuration block in Config.groovy:

ignite {
    enabled=true
    config.locations = [
            "file:ignite/conf/*.groovy"
    ]
    gridName="myGrid"
    
    /*
     * This setting must be enabled on the machine that builds the WAR file for the target environment,
     * since it will determine if the correct filters are incorporated into the web.xml file
     */
    webSessionClusteringEnabled=true
    
    /*
     * Configure timeouts for long-running jobs. If not specified, defaults to 60000
     */
    defaultJobTimeout = 60000
    
    /**
     * Set of Ant paths to exclude from web session clustering logic
     */
    webSessionClusteringPathExcludes = ['*/api/**','*/otherExclude/']
    
    /** 
      * Enable distributed hibernate l2 caching
      * You must also set the region factory correctly 
      */
    l2CacheEnabled=true
    
    /**
     * DEFAULTS; you can also configure individual caches as spring beans 
     */
    l2Cache {
        entity {
            evictionPolicy = 'lru'
            cacheMode = CacheMode.PARTITIONED
            memoryMode = CacheMemoryMode.OFFHEAP_TIERED
            atomicityMode = CacheAtomicityMode.TRANSACTIONAL
            writeSynchronizationMode = CacheWriteSynchronizationMode.FULL_ASYNC
            offHeapMaxMemory = (1024L * 1024L * 1024L) * 0.25;
            maxElements = 1000
            swapEnabled = false
            statisticsEnabled = true
            managementEnabled = true
        }
        association {
            evictionPolicy = 'lru'
            cacheMode = CacheMode.PARTITIONED
            memoryMode = CacheMemoryMode.OFFHEAP_TIERED
            atomicityMode = CacheAtomicityMode.TRANSACTIONAL
            writeSynchronizationMode = CacheWriteSynchronizationMode.FULL_ASYNC
            offHeapMaxMemory = (1024L * 1024L * 1024L) * 0.25;
            maxElements = 1000
            swapEnabled = false
            statisticsEnabled = true
            managementEnabled = true
        }
    }
    
    peerClassLoadingEnabled=false
    discoverySpi {
        networkTimeout = 5000
        addresses = ["${myIP}:47500..47509"]
    }
}

The files can be located anywhere but in the example above we have put them under ignite/conf in the project root. In a production deployment they would likely be in a completely different directory.

The configuration files follow the standard Ignite spring configuration conventions, however they must (for the time being) be expressed as Grails Spring DSL files for use with a BeanBuilder.

See the ignite/conf directory for sample configuration files. For basic configuration you can copy the directory to your project.

Startup

In order to facilitate the correct startup order of the Ignite grid (to support hibernate L2 caching it must start before the SessionFactory, which in turn is started by Grails before the application), Ignite uses a special startup hook called IgniteStartupHelper. The IgniteStartupHelper class will load it's own application context specifically for the Ignite grid. This application context can be managed using the Spring DSL by specifying beans in *Resources.groovy files in the ignite/conf directory of your application, or by specifying external config directories for these files:

ignite {
    enabled=true
    config.locations = [
            "file:ignite/conf/*.groovy"
    ]
    gridName="myGrid"

Discovery

Ignite can be configured to discover nodes via static, multicast or s3 discovery mechanisms. You need to include the correct ignite spring resources in your project and set the proper Grails configuration.

See IgniteResources.groovy

The example below shows how to configure discovery via the Grails configuration:

    discoverySpi {
        s3Discovery = true 
        awsAccessKey = <ACCESS KEY WITH S3 PERMISSIONS>
        awsSecretKey = <SECRET KEY FOR ACCESS KEY>
        s3DiscoveryBucketName = "ignite-localdev"

		/* OR */
		
        multicastDiscovery = false
        networkTimeout = 5000
        addresses = ["${myIP}:47500..47509".toString()]

		/* OR */
		
        // use static IP discovery
        addresses = ['1.2.3.4','5.6.7.8']
    }

Logging

The project contains an implementation of IgniteLogger for use with Grails. This class allows you to use the Grails log4j DSL to configure logging for the embedded Ignite node. The logger can be configured from the Ignite spring bean:

        gridLogger(org.grails.ignite.IgniteGrailsLogger)
        
        igniteCfg(IgniteConfiguration) {
        	gridLogger = ref('gridLogger')
        }

Distributed Hibernate L2 Caching

Requires Hibernate 4

A basic functional version of distributed Hibernate L2 caching can be utilized by setting the region factory class as follows:

hibernate {
    cache.region.factory_class = 'org.grails.ignite.HibernateRegionFactory'
    org.apache.ignite.hibernate.grid_name = '<MY GRID NAME>'
    org.apache.ignite.hibernate.default_access_type = 'READ_ONLY' // see Ignite docs
}

By default, the plugin will create caches with reasonable defaults (whatever defaults existing when using Ignite.getOrCreateCache()) on demand when Hibernate configures the regions. You can override these defaults by creating the appropriate CacheConfiguration beans in IgniteCacheResources.groovy

##Example Spring Cache Configuration

In grails-app/ignite/conf/resources/IgniteCacheResources.groovy:

    'com.mypackage.MyDomainClass' { bean ->
        name = 'com.package.MyDomainClass'
        cacheMode = CacheMode.PARTITIONED
        atomicityMode = CacheAtomicityMode.TRANSACTIONAL
        writeSynchronizationMode = CacheWriteSynchronizationMode.FULL_SYNC
        evictionPolicy = { org.apache.ignite.cache.eviction.lru.LruEvictionPolicy policy ->
            maxSize = 1000000
        }
    }

See Also:

http://apacheignite.gridgain.org/v1.1/docs/evictions

Scheduled, Distributed Tasks

This plugin provides an Ignite service called DistributedSchedulerService that provides a partial implementation of the ScheduledThreadPoolExectutor interface but allows you to run the submitted jobs on the Ignite grid.

The methods scheduleAtFixedRate and scheduleWithFixedDelay are currently implemented. The service keeps track of submitted job schedules using a grid-aware Set that is configured for REPLICATED caching, so that if any grid node goes down.

A Grails service of the same name ("DistributedSchedulerService") is also provided to facilitiate easy injection into other Grails applications.

Example

distributedSchedulerService.scheduleAtFixedRate(new HelloWorldGroovyTask(), 0, 1000, TimeUnit.MILLISECONDS);

This example shows how to schedule the supplied task to execute once per second on the entire grid, regardless of the grid topology. The execution will be evenly load-balanced across all grid nodes. If any grid nodes go down the rebalancing will result in the same execution rate (once per second in this example).

The example above can be run out-of-the-box (the HelloWorldGroovyTask is included in the plugin). You can then try neat things like spinning up another instance on a different port, and watching the grid fail-over and recover by killing one instance and bringing it back up.

Notes

Requires h2 version 1.3.137 (or higher)? Make sure you do this:

inherits("global") {
    // specify dependency exclusions here; for example, uncomment this to disable ehcache:
    // excludes 'ehcache'
    excludes 'h2'
}

grails-ignite's People

Contributors

dstieglitz avatar joshuacwebdeveloper avatar robertoschwald avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

grails-ignite's Issues

Unable to run tests: NoClassDefFoundError: org/h2/result/RowFactory

This issue is being created with the intention of submitting a pull request for it.

grails test-app exists with the following error (before running any tests):

2018-12-19 14:45:29,373 [main] INFO  ignite.IgniteStartupHelper  - [grails-ignite] Starting Ignite grid...
Error |
Fatal error running tests: Error creating bean with name 'transactionManagerPostProcessor': Initialization of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager': Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory': Invocation of init method failed; nested exception is java.lang.NoClassDefFoundError: org/h2/result/RowFactory (Use --stacktrace to see the full trace)
.Tests FAILED
|
 - view reports in C:\Users\Joshua\Documents\WebDev\hapn.in\grails-ignite\target\test-reports
Error |
Forked Grails VM exited with error

Looks like the h2 dependency is missing.

Create MessagingService.deregisterReceiver()

In MessagingService.registerReceiver(), return the UUID from IgniteMessaging.remoteListen();

Add a deregisterReceiver() method to MessagingService. This method could merely wrap around IgniteMessaging.stopRemoteListen() by passing through the above UUID.

l2 cache test is failing

PR #9 fixes most of the failing tests; however, it leaves one test failing, as it is unclear whether l2 cache was supposed to be working or not, and it is an experimental feature.

Figure out why this test is failing:

test l2 cache configuration

Executed in 1.243 seconds.

Condition not satisfied: grid.cache('org.grails.ignite.Widget').size() == 1 | | | | | | 0 false | org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy@646ddc20 org.grails.ignite.IgniteContextBridge@5a941fb4

junit.framework.AssertionFailedError: Condition not satisfied:

grid.cache('org.grails.ignite.Widget').size() == 1
|    |                                 |      |
|    |                                 0      false
|    org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy@646ddc20
org.grails.ignite.IgniteContextBridge@5a941fb4

	at org.grails.ignite.ConfigurationIntegrationSpec.test l2 cache configuration(ConfigurationIntegrationSpec.groovy:61)
System output

[atomicCache, transactionalCache, org.hibernate.cache.spi.UpdateTimestampsCache, org.grails.ignite.Widget, test_partitioned, test_replicated, org.hibernate.cache.internal.StandardQueryCache]
org.grails.ignite.Widget : 1
Statistics[start time=1545422884556,sessions opened=1,sessions closed=0,transactions=2,successful transactions=0,optimistic lock failures=0,flushes=0,connections obtained=3,statements prepared=2,statements closed=0,second level cache puts=0,second level cache hits=0,second level cache misses=1,entities loaded=1,entities updated=0,entities inserted=1,entities deleted=0,entities fetched=0,collections loaded=0,collections updated=0,collections removed=0,collections recreated=0,collections fetched=0,naturalId queries executed to database=0,naturalId cache puts=0,naturalId cache hits=0,naturalId cache misses=0,naturalId max query time=0,queries executed to database=0,query cache puts=0,query cache hits=0,query cache misses=0,update timestamps cache puts=0,update timestamps cache hits=0,update timestamps cache misses=0,max query time=0]
[org.grails.ignite.Widget:SecondLevelCacheStatistics[hitCount=0,missCount=1,putCount=0,elementCountInMemory=0,elementCountOnDisk=-1,sizeInMemory=-1]]

Cannot get Log4J logging to work

Using a spring- or programmatic-configuration, I am unable to install the Log4JLogger due to a ClassNotFoundException. When trying to run the application in non-forked mode, another ClassLoader exception prevents the application from running.

In forked mode:

java.lang.NoClassDefFoundError: org/apache/ignite/IgniteLogger
    at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
    at IgniteGrailsPlugin$_closure2.doCall(IgniteGrailsPlugin.groovy:130)
    at grails.spring.BeanBuilder.invokeBeanDefiningClosure(BeanBuilder.java:754)
    at grails.spring.BeanBuilder.beans(BeanBuilder.java:584)
    at grails.spring.BeanBuilder.invokeMethod(BeanBuilder.java:527)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.ClassNotFoundException: org.apache.ignite.IgniteLogger
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 18 more

In non-forked mode:

60397 [localhost-startStop-1] ERROR org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal  - Heuristic transaction failure.
class org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: Failed to locally write to cache (all transaction entries will be invalidated, however there was a window when entries for this transaction were visible to others): GridNearTxLocal [nearLocallyMapped=false, colocatedLocallyMapped=true, hasRemoteLocks=false, mappings=[524c22d1-92be-4afe-96d6-6d7f7ed72c69], super=GridDhtTxLocalAdapter [dhtThreadId=40, needsCompletedVers=false, nearNodes=[]... 

(truncated due to length)

Can't configure Ignite L2 cache due to spring bean startup order

In order to have a working Ignite L2 cache, the Ignite node needs to be started before the Hibernate SessionFactory is configured. Currently, Grails will configure the Hibernate SessionFactory before Ignite starts and therefore if the Ignite cache region factory is configured in DataSource.groovy, it will fail since it's looking for a started node.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.