AWS useful articles

 Lambda with RDS 

     https://www.jeremydaly.com/reuse-database-connections-aws-lambda/

     https://www.jeremydaly.com/manage-rds-connections-aws-lambda/

 Good part:

     database connection management recommendation

 Added Value:

    https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/rds-proxy.html

   (This seems like a DB connection pool management)

SQL vs NOSQL (DynamoDB)

This is not about the detailed comparison between SQL and NOSQL.  There are a lot of articles online already regarding this topic.
 This is about my experience with SQL and NOSQL.
 I primarily used Oracle, Mysql and some DB2.  And I am mainly using Aurora SQL database and  DynamoDB NOSQL nowadays.
 What I learned:
    Even though a lot of online talks mentioned using one single table approach with DynamoDB for microservices, it is not easy to do, especially if a microservice has relatively complex domain and goes through a lot of changes.

    DynamoDB case insensitive search is not straightforward; it takes extra effort compared to SQL. When data needs to be encrypted and searched, be careful about the data management (for example, use lower case, use hash).
    DynamoDB throttling can be a pain depending on how you manage it, or whether your company is willing to pay more to avoid throttling.
    DynamoDB global secondary indexes could become expensive if you have too many.
    Some DynamoDB tricks mentioned online, for example, using some fake hashkey to enforce unique key constraints, using the same column to store many different types of data, could become a nightmare for application maintenance.  Not many developers will be able to understand the code easily.
   DynamoDB transaction apis could provide great value to developers that come from the SQL world. But the transaction APIs are more expensive and relatively slow.
   Even with DynamoDB, you may still need to manage some kind of relationships between entities, which basically is similar to what you would do with SQL DB.
   DynamoDB is really a dummy map.  Some things you normally can do with SQL DB, for example, create timestamp, SEQUENCE, becomes a burden in application code.
   DynamoDB stream is actually a great feature. But when integrating with lambdas, some duplication processing could happen in some cases.
   SQL database is not bad when it comes to handling large volume. Facebook mysql use is a good example. It takes good design, tuning, and maybe even customization.

 

Machine Learning

Machine Learning

Purpose
    Machine Learning is to generalize.

Classic Problem
    Normal Programming: "Hello world"
    Machine Learning:  MNIST

Approach
    Problems --> Tools--->Metrics  (apply to all problems?)
    Data to generalize --> Use different algorithms --> Monitor performance of algorithms and adjust

Key Words
     Classification
        Discrete output
     Regression
        Continuous numeric output
    Clustering
 
     Gradient descent, Backpropagation, Cost function,
     Cross-entropy
           Any loss consisting of a negative log-likelihood between the empirical distribution
           defined by the training set and the probability distribution defined by model. For example,
           Mean Squared Error: cross-entropy between empirical distribution and a Gaussian model

     Activation function
           Step function
                discrete 0, 1
           Sigmoid function
             
           Tanh function

           Rectified Linear function (ReLU)

           Exponential linear unit (ELU)

     Training data set
           Train parameter
     Validation data set
           Tune Hyperparameter
     Test data set
   
     Bias, Variance
         Linked to capacity, underfitting, overfitting

     Closed-form solution

     Parameter
            Learned
            Weight, Bias
     HyperParameter
           Tuned
           Learning rate
           number of layers
           number of nueons each layer
           number of iterations
       
     Accuracy
     Sensitivity
     Specificity
     F1-score

    Kernel trick
    Maximum likelihood estimation
             Point estimate of variables

     Bayesian estimation
             Full distribution of variables
 
    Optimization
         Hill Climbing
              One step along axis one time
              Achieve Optimal solution for Convex problem
              Problems: local maxima, ridges and alleys, plateau
              Good for function complex and/or not differentiable

         Gradient Descent
             Vanishing/exploding gradients problems
             approaches to solve: He initialization, Batch Normalization

         Momentum

          AdaGrad

          RMSProp

          Adam
       
    Regularization
        Modification to ML algorithms, intending to reduce generalization error, not training error
        Example: weight decay for linear regression
        Early stoppping, L1, L2, Dropout, Max-Norm, Data Augmentation

     Generalize
            To have small gap between training error and test error
     Supervised Learning
            features + labels
            Nonprobabilistic SL
                  K-Nearest Neighbor
             Decision Tree
     Unsupervised Learning
            features without labels
     Reinforcement Learning
             Learning by getting feedback from the environment

     Transformer:
         Modify or filter data before feeding it to learning algorithms
         Preprocessing
         Feature selection
         Feature extraction
         Dimension reduction (PCA, manifold learning)
         Kernel approximation

    Cross-validation schemes
         K-fold
         Stratified K-fold
         Leave-one-out (small amount of data)

    Dimension Reduction
         PCA
         KPCA
         LLE
 
Math behind ML
      z=wx+b
    σ(z)=1/(1+e−z)
    ...


Concepts
     Model--Train--Evaluate--Predict
     Classification, Regression, Clustering, Dimension deduction

Algorithms
    Linear Regression
        Find optimal weights by solving normal equations

    Logistic Regression
         No closed-form solution. Maximizing the log-likelihood, or minimizing the negative log-likelihood using gradient descent.

    Neural Network
    RNN (Recurrent Neural Network)
    CNN (Convolutional Neural Network)

    Decision Tree

    Identification Tree

    Naive Bayes
           Features independent of each other
           Conditional Probability Model
           Highly scalable, only requires small amount of training data
           Linear Performance Time
           Generally outperformed by other algorithms, SVM...

    Support Vector Machines
           For both classification and regression
           Widest street to separate instances of different classes

    Random Forest
         Decision Tree ensemble

Test Methodologies
   Leave one out   LOO
       for small amount of data

   Data split (80/20)
     
Practical Guidelines for DNN
   Initialization                        He
   Activation                           ELU
   Normalization                     Batch Normalization
   Regularization                    Dropout
   Optimizer                           Adam
   Learning Rate Schedule     None

Software
    Tensorflow, Scikit-learn
    Spark MLLib, Spark ML,  Weka,


Use cases
    Linear Regression
          House size---> House price in a community
 
    Naive Bayes
          Document classification: separate legitimate emails from spam emails
          For example, based on key words: cheap, free
     
Questions
       When to use which algorithm(s)?

Classic Applications
       Alphago vs Lee Sedol
       https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol

       Autonomous Car

       Netflix movie recommendations

       Image recognitions

       Natural language processing

Summary
     No ML algorithm is universally better than any other algorithm.
     Understand data distribution, and pick proper algorithm(s).

References
 
    Books
          (One of my favorite books, highly recommended)
           Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems

          Algorithms for Reinforcement Learning

           Deep Learning (Adaptive Computation and Machine Learning series)

           http://neuralnetworksanddeeplearning.com/

    TensorFlow

    scikit-learn

    https://www.kaggle.com/

    Reinforcement Learning - David Silver

   http://www.wildml.com/

    https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-034-artificial-intelligence-fall-2010/

    Machine learning series from Luis Serrano  (Very good explanations for beginners)
    https://www.youtube.com/watch?v=aDW44NPhNw0
    https://www.youtube.com/watch?v=BR9h47Jtqyw&t=24s
    https://www.youtube.com/watch?v=2-Ol7ZB0MmU&t=7s
    https://www.youtube.com/watch?v=IpGxLWOIZy4

    http://scikit-learn.org/stable/tutorial/machine_learning_map/

    https://blogs.sas.com/content/subconsciousmusings/2017/04/12/machine-learning-algorithm-use/
         
    https://s3.amazonaws.com/assets.datacamp.com/blog_assets/PythonForDataScience.pdf

   https://s3.amazonaws.com/assets.datacamp.com/blog_assets/Scikit_Learn_Cheat_Sheet_Python.pdf

    https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf

    (AWS machine learning service)
    https://aws.amazon.com/blogs/aws/sagemaker/

    (Spark MLlib example)
    https://stanford.edu/~rezab/sparkworkshop/slides/xiangrui.pdf

    https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-0002-introduction-to-computational-thinking-and-data-science-fall-2016/

    https://biomedical-engineering-online.biomedcentral.com/articles/10.1186/s12938-017-0378-z

    https://iknowfirst.com/rsar-machine-learning-trading-stock-market-and-chaos

    Mastering the game of Go without human knowledge

 

String valueOf() pitfalls

What will the console output of this program?


public class TestStringValueOf {

public static void main(String[] args) {
testStringValueOfChar();
}


      public static void testStringValueOfChar() {
char a = 'a';
String str1 = String.valueOf(a);
String str2 = String.valueOf(a);
System.out.println("char comparison:" + (str1 == str2));


double d = 12.3d;
String str3 = String.valueOf(d);
String str4 = String.valueOf(d);
System.out.println("double comparison:" + (str3 == str4));


boolean b = false;
String str5 = String.valueOf(b);
String str6 = String.valueOf(b);
System.out.println("boolean comparison:" + (str5 == str6));


Object o = null;
String str7 = String.valueOf(o);
String str8 = String.valueOf(o);
System.out.println("Object null comparison:" + (str7 == str8));


Object notNull = new Object();
String str9 = String.valueOf(notNull);
String str10 = String.valueOf(notNull);
System.out.println("Object Not null comparison:" + (str9 == str10));
  }
}

see the end of this article for the output.

Overall, the string comparison should use 'equals' no matter how String objects were created.


-------console output----------

char comparison:false
double comparison:false
boolean comparison:true
Object null comparison:true
Object Not null comparison:false

Spring MVC UTF-8

Key points

web.xml:
     <filter>
    <filter-name>encodingFilter</filter-name>
    <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
    <init-param>
       <param-name>encoding</param-name>
       <param-value>UTF-8</param-value>
    </init-param>
    <init-param>
       <param-name>forceEncoding</param-name>
       <param-value>true</param-value>
    </init-param>
</filter>
<filter-mapping>
    <filter-name>encodingFilter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>


Maven pom.xml:
  <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
       ...
  </properties>

JSP:
   <%@ page language="java" pageEncoding="UTF-8"%>
  <%@ page contentType="text/html;charset=UTF-8" %>

Compile xsl files and store in cache to improve XSLT performance

Common code found online to do XSLT transformation. (removed non essential pieces for brevity)

------------------

TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer(new StreamSource(new File(xsltPath)));
transformer.transform(new StreamSource(new File(sourceFilePath)), new StreamResult(new File(resultPath)));

----------------

The code works. But if a xslt file is relatively big and  

needs to be used over and over again to transform 

a lot of files, for example, in the batch mode, 

it may not perform well. 

The following shows a way to cache the compiled version of an xsl file, which is a 'Templates' object. This object is thread safe.

Code snippet to cache the 'Templates' object.

static final Map<String, Templates> cacheTemplates = new ConcurrentHashMap<String, Templates>();

       static TransformerFactory transformFactory = null;

       static {

             init();

       }

     private static void init() {

         try {

             transformFactory =TransformerFactory.newInstance();

        }

        catch(Exception e) {

            throw new RuntimeException(e);

        }

    }

     public static void cacheCompiled( String xsl) {
File file = null;
  StreamSource source= null;
                Templates  templates = null;
try {
file = new File( xsl);
source = new StreamSource(file);
                         templates = transformFactory.newTemplates(source); //create this once for a file, save in a cache.
cacheTemplates .put(xsl, templates );
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
}
}

The above 'templates' object is basically a coompiled version of the original xsl file.  If the original file is relatively big, for example, 20KB, it takes more than 2 seconds on my local machine to transform a small file.  Without caching the templates, it takes more than 2 seconds every time.  With caching,  it takes about 0.1 seconds  for every transformation after the first time.



The basic code is like this:

//get the Templates object from cache based on the xsl file name, then get a Transformer object

Transformer transformer = templates.newTransformer();

transformer.transform(new StreamSource(new File(sourceFilePath)),
new StreamResult(new File(resultPath)));

The 'transformer' object mentioned above is not thread safe.

The SAXON parser seems becoming more popular, and the Xalan parser seems fading away.

The home edition of the SAXON parser, which is free, may be good enough for a lot of applications.

First impressions on open source ESBs

Used commercial ESB and BPMs for a couple of years, recently had a chance to evaluate some open source ESBs.

WSO2:  not easy to use, had difficulty even making the sample projects to work. No DataMapper tool, which is a big no-no to my projects.

Mulesoft ESB:  Nice documentation, instructions easy to follow, sample projects can be built and run in a couple of minutes, nice DataMapper tool in the 3.4 version.   Have not had a chance to build a relatively complex application using this.  Not sure whether the community edition is good enough to be used in the Production.

String getBytes could lead to difficult bugs

If you execute the following function,  what do you think should be the size of the 'def' byte array?

The logic is really simple: an input as byte array that have two elements, then create a string out of this with 'UTF-8' encoding, then create another byte array using this string with the same UTF-8 encoding.

public static void testStringUTF8() {
byte[] abc = new byte[2];
abc[0] = 31;
abc[1] = -117;

try {
String stringAbc = new String(abc, "UTF-8");
byte[] def = stringAbc.getBytes("UTF-8");
if (def != null) {
System.out.println("size of output byte array:" + def.length);  //print the array size
}

System.out.println(def[1]);  //print the second element of the output byte array

System.out.println(abc[1]); //print the second element of the input byte array

} catch (Exception e) {
e.printStackTrace();
}
}

---------------

How to invoke local EJB session beans in WebLogic

Sometimes you may have a need to invoke a LOCAL EJB session beans in a normal java class, for example, Business Delegate class, you can use ServiceLocator to locate a local EJB session bean proxy by JNDI name. Even though it is relatively easy to do so for a REMOTE EJB session bean by using the value of  'name' or 'mappedName' in the bean class definition, it is a little tricky for LOCAL session beans.

Here is what you need to do.

For exampe:

Here is an interface:

package  com.play;

@Local
public interface PlayFacadeInf {
     public void play(String var);
}


Here is the implementation bean class.

package  com.play;

@Stateless
public class  PlayFacadeImpl implements  PlayFacadeInf {
     public void play(String var) {
          //...do somthing
    }
}


Here is the part of the ejb-jar.xml


display-name>myEJB </display-name>
  <enterprise-beans>
<session>
<ejb-name> PlayFacadeImpl</ejb-name>
<ejb-class>com.play.PlayFacadeImpl</ejb-class>
<ejb-local-ref>
<ejb-ref-name>ejb/PlayFacadeInf</ejb-ref-name>
<ejb-ref-type>Session</ejb-ref-type>
<local>com.play.PlayFacadeInf</local>
</ejb-local-ref>
</session>
   </enterprise-beans>

Here is part of web.xml


<ejb-local-ref>
<ejb-ref-name>ejb/PlayFacadeInf</ejb-ref-name>
<ejb-ref-type>Session</ejb-ref-type>
<local>com.play.PlayFacadeInf</local>
</ejb-local-ref>

Here is part of the ServiceLocator.java



private static InitialContext ctx = null;
static {
try {
ctx = new InitialContext();
}
catch (NamingException e) {
//... throw some exception
}
}

private static InitialContext getInitialContext() throws NamingException{
return ctx;
}


public static  PlayFacadeInf  getPlayFacade() throws NamingException {

PlayFacadeInf     playFacadeInf   = null;

playFacadeInf     = ( PlayFacadeInf )            
                       ServiceLocator.getInitialContext().lookup("java:/comp/env/ejb/PlayFacadeInf");

return  playFacadeInf;
}

Then any normal java class can use the ServiceLocator to get hold of the local ejb session bean proxy.

JPA NoResultException marks transaction rollback in WebLogic 10.3.2

The weblogic 10.3.2 server comes with two jpa implementations: eclipselink (org.eclipse.persistence_1.0.0.0_1-2-0.jar) and openjpa (org.apache.openjpa_1.0.1.0_1-1-1-SNAPSHOT.jar).  By default, it uses the openjpa. But you can add “<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>” to the persistence.xml to use the eclipselink implementation. 

Both implementations do not handle “NoResultException” properly. The “NoResultException”thrown in a transaction would mark the transaction as rolledback, which violates the JPA specification.

Some sample code (using EJB 3) is as follows:

@Stateless

public class FacadeImpl implements FacadeInf {

                        @EJB

                        DAOInf dAOInf;

                        public void findOrCreate(Long [] addressIds, PersonAddressData personAddressData) {

                        ..........

for (Long addressId: addressIds) {

try {

                                                                                                dAOInf.findByAddress(addressId);

}

catch(Exception e) { //find failed, try to create something

                                                                                                ....createInfo(personAddressData);

}

                                                }

                        }

}

@Stateless

public class DAOImpl implements DAOInf {

                        @PersistenceContext(unitName = "SampleJPA")

                        EntityManager em;

                        //@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)

                        public Person findByAddress(Long addressId) {

                                                return (Person)em.createNamedQuery("someNamedQuery").getSingleResult();

                        }

}

In the above snippet,  if in the “for loop”, one invocation of “findByAddress” threw the “NoResultException”, the active transaction would be marked as rolledback, the “createInfo” in the “catch” block would not be able to accomplish anything. But if “@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)” n the above snippet, if specification.edbackdd pselinkLogic 10.3.2

were not commented out, which means the “findByAddress” would run in a “NON transaction” context, in this case, even if the “find” function failed, the transaction could still continue, so the “createInfo” could be executed properly.

Most find functions (except findByPrimaryKey) in session bean DAOs should be marked as “Transaction NOT SUPPORTED” (@TransactionAttribute(TransactionAttributeType.NOT_SUPPORTED)).

How to Fix Java POJO Annotations

POJO and Annotation are a big part of the java world nowadays.  Spring, EJB, JPA, Servlet,  JSF and other technologies are using annotated POJOs. The traditional definition is that a POJO is an ordinary Java Object, which means it has no dependency on any framework or container.   

But take a look at the following example:

import javax.ejb.*;

@Stateless(name = "Test11", mappedName = "ejb.abc")

public class Test11Bean implements Test11{

      private static final long serialVersionUID = -128L;

  @EJB

  protected TestSb testsb;

  public Double getCost(String name, String expedite) {

      return 0.0;

}

….

}

As we can see, without ejb related jar file in the classpath, this code would not compile at all. If we want to use this in a weblogic container eventually, we may add “TransactionTimeoutSeconds” and other weblogic kodo related annotations.  In this case, we need to have vendor-specific jar files in order to compile this class. 

I think some changes can be done to make it a traditionally-defined POJO.  Here is what I would do:

I would change the “import” to “@import” (or maybe another new keyword.) This will serve as a hint during the compilation, deployment and run times. During different stages (compilation, deployment and run), depending on whether the related jar files are in the classpath, different actions can be taken to generate different artifacts.

What are the benefits of doing this?

1.       Code can be reused more often.  Sometimes I want to use a JPA POJO entity class as a pure simple bean (like a data transfer object) in another project,  it would be possible if it did not have dependency on some jars.

2.       Could make the testing easier.

How to use EJB 3 timer in a weblogic 10 cluster environment

Although there are some articles talking about EJB 3 timers or job schedulers , we were not able to find any detailed instructions on how to make EJB 3 timer work in a weblogic cluster. In this article we will go through a sample project to show how EJB 3 timers are used in a weblogic cluster. The sample project will create two recurring timers, the first recurring timer will periodically print out some simple information, the second recurring timer will create a couple of one-timer timers, each of which will print out some information. In this article, we will show you how to use weblogic admin console to configure the cluster, how the application uses the related configuration from the console and how to invoke timers, also explain what problems we faced and how we solved them.

Environment:   

web logic server 10.3.2, oracle database 11gR1, Eclipse Helios

Code:

Timer1SessionBeanLocal: local interface for creating timer

@Local

public interface Timer1SessionBeanLocal {

      public void createTimer();

}

Timer1SessionBean: a recurring timer that prints out something

@Stateless

public class Timer1SessionBean implements Timer1SessionBeanLocal {

      @Resource

      TimerService timerService;

    public Timer1SessionBean() {

    }

    public void createTimer() {

            timerService.createTimer(

                        60000,

                        60000, null);

      }

   

      @Timeout

      public void timeout(Timer arg0) {

            System.out.println("recurring timer1 : " + new Date());

      }

}

Timer2SessionBean: a recurring timer that creates a bunch of one-time timers,also the number of one-time timers created is roughly based on the maximum allowed number of active timers minus the number of active timers at that time.

@Stateless

public class Timer2SessionBean implements Timer2SessionBeanLocal {

      @Resource

      TimerService timerService;

      @EJB

      Timer3SessionBeanLocal timer3Bean;

      public Timer2SessionBean() {

      }

      public void createTimer() {

            timerService.createTimer(120000, 300000, null);

      }

      @Timeout

      public void timeout(Timer arg0) {

            System.out.println("recurring timer2 : " + new Date());

            // used to control the total number of threads running in the app

            // use 10 as maximum in this example.

            int numberOfActiveTimers = timer3Bean.getCountOfActiveTimers();

            if (numberOfActiveTimers < 10) {

                  int toCreateNum = 10 - numberOfActiveTimers;

                 

                  for (int i = 0; i < toCreateNum; i++) {

                        Timer3Info info = new Timer3Info();

                        // set start delays to be 30,60,90... seconds

                        info.setDelay(30000 * (i + 1));                                                                      

                        timer3Bean.createTimer(info);

                  }

            }

            System.out.println("Exit timeout in timer2");

      }

}

Timer3SessionBean: one-time timer created by another timer, provides the number of active timers for this bean, and prints out something.

@Stateless

public class Timer3SessionBean implements Timer3SessionBeanLocal {

      @Resource

      TimerService timerService;

      public Timer3SessionBean() {

      }

      public void createTimer(Timer3Info timerInfo) {

            timerService.createTimer(timerInfo.getDelay(), null);

      }

      @Timeout

      public void timeout(Timer arg0) {

            System.out.println("one-time timer3 : " + new Date());

      }

      /**

       *

       * @return the number of active timers

       */

      public int getCountOfActiveTimers(){

            int retVal = 0;

            try {

                  //In rare occasions, could throw NullPointerException //because of a bug in weblogic

                  @SuppressWarnings("unchecked")

                  Collection<Timer> timersCol = timerService.getTimers();

                 

                  if (timersCol != null)

                        retVal = timersCol.size();

            } catch (Exception e) {

                  //if it failed, use the maximum (10 in this example), so no //new timers can be created

                  retVal = 10;

            }

            return retVal;

           

      }

}

TestTimerCreateServlet: used to create recurring timers.

public class TestTimerCreateServlet extends HttpServlet {

      private static final long serialVersionUID = 1L;

      @EJB

      Timer1SessionBeanLocal timer1;

      @EJB

      Timer2SessionBeanLocal timer2;

      /**

       * @see HttpServlet#HttpServlet()

       */

      public TestTimerCreateServlet() {

            super();

      }

      /**

       * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse

       *      response)

       */

      protected void doGet(HttpServletRequest request,

                  HttpServletResponse response) throws ServletException, IOException {

            System.out.println("start timer creation : " + new Date());

            try {

                  timer1.createTimer();

                  timer2.createTimer();

            } catch (Exception e) {

                  System.out.println("timer creation failed ");

                  throw new  RuntimeException("timer creation failed ", e);

            }

            System.out.println("Done timer creation : ");

      }

      /**

       * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse

       *      response)

       */

      protected void doPost(HttpServletRequest request,

                  HttpServletResponse response) throws ServletException, IOException {

            doGet(request, response);

      }

}

Overall, the code is quite simple. Some things are worth noting here is that local interfaces are used for the session beans, a servlet which needs to be invoked externally is used to create timers, also ‘timerService.getTimers()’ is used to find out the number of active timers for a session bean and also help control the number of running timers in the system, so the system will not be over stretched.

weblogic-ejb-jar.xml: some important configurations

 

<?xml version="1.0" encoding="UTF-8"?>

<wls:weblogic-ejb-jar xmlns:wls="http://xmlns.oracle.com/weblogic/weblogic-ejb-jar" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/ejb-jar_3_0.xsd http://xmlns.oracle.com/weblogic/weblogic-ejb-jar http://xmlns.oracle.com/weblogic/weblogic-ejb-jar/1.0/weblogic-ejb-jar.xsd">

    <!--weblogic-version:10.3.2-->

    <wls:weblogic-enterprise-bean>

        <wls:ejb-name>Timer1SessionBean</wls:ejb-name>

        <wls:stateless-session-descriptor>

            <wls:timer-descriptor>

                <wls:persistent-store-logical-name>timerst</wls:persistent-store-logical-name>

            </wls:timer-descriptor>

        </wls:stateless-session-descriptor>

    </wls:weblogic-enterprise-bean>

    <wls:weblogic-enterprise-bean>

        <wls:ejb-name>Timer2SessionBean</wls:ejb-name>

        <wls:stateless-session-descriptor>

            <wls:timer-descriptor>

                <wls:persistent-store-logical-name>timerst</wls:persistent-store-logical-name>

            </wls:timer-descriptor>

        </wls:stateless-session-descriptor>

    </wls:weblogic-enterprise-bean>

    <wls:weblogic-enterprise-bean>

        <wls:ejb-name>Timer3SessionBean</wls:ejb-name>

        <wls:stateless-session-descriptor>

            <wls:timer-descriptor>

                <wls:persistent-store-logical-name>timerst</wls:persistent-store-logical-name>

            </wls:timer-descriptor>

        </wls:stateless-session-descriptor>

    </wls:weblogic-enterprise-bean>

    <wls:timer-implementation>Clustered</wls:timer-implementation>

</wls:weblogic-ejb-jar>

The most important things are that making sure the timers are cluster aware, and also using proper logical name for the persistent store  (timerst), which will be configured in the administrator console.

Admin console configuration:

Since EJB timers will be running in a clustered environment, weblogic uses two tables managing timers: ACTIVE and WEBLOGIC_TIMERS. These two tables can be named differently. These tables can be created automatically by the weblogic or can be created by you manually. Before you configure the cluster, you need to create JDBC data sources. 

Persistence of timers

There are two different styles of persistence. One is file based, the other is database based. We used the database for persistence. 

A table called ABC_WLSTORE will be created in schema ABCSYS automatically by the weblogic. It can also be created manually. 

The logical name (timerst) mentioned in the above screen shot is  (must be) exactly the same as the value of ‘persistent-store-logical-name’  in the weblogic-ejb-jar.xml. In our application we had two nodes in a cluster. The important lesson we learned was that creating only one persistence store that targets one migratable, as shown above, is the right thing to do. Do NOT create two persistent stores with each targeting one migratable. Do NOT create different persistent stores with the same data source and the same prefix name.

Deploy and Run

After deploying the application, you can invoke http://server:port/context/TestTimerCreateServlet to create timers, and then you can monitor the log files and/or the data in the WEBLOGIC_TIMERS table to find out how the execution went. 

Lessons learned

PostConstruct :  

We tried using a “PostConstruct” function in a session bean to create timers and had hoped that timers would be created when a session bean is deployed, but it failed. The sample code was

@PostConstruct

public void init() {

timerService.createTimer(60000,60000, null);

}

Exceptions:

Exception after create timer : java.lang.IllegalStateException: [EJB:010193]Illegal call to EJBContext method. The bean is in "ejbCreate" state. It cannot perform null action(s). Refer to the EJB specification for more details.
java.lang.IllegalStateException: [EJB:010193]Illegal call to EJBContext method. The bean is in "ejbCreate" state. It cannot perform null action(s). Refer to the EJB specification for more details.
at weblogic.ejb.container.internal.BaseEJBContext.checkAllowedMethod(BaseEJBContext.java:147)
at weblogic.ejb.container.internal.BaseEJBContext.checkAllowedToUseTimerService(BaseEJBContext.java:439)
at weblogic.ejb.container.internal.TimerServiceImpl.createTimer(TimerServiceImpl.java:82)
at weblogic.ejb.container.internal.TimerServiceImpl.createTimer(TimerServiceImpl.java:43)
at weblogic.ejb.container.deployer.TimerServiceProxyImpl.createTimer(TimerServiceProxyImpl.java:60)

Reason given by weblogic:

From the EJB 3.0 Specification, page 84, http://www.jcp.org/en/jsr/detail?id=220

The following steps describe the life cycle of a stateless session bean instance:

- A stateless session bean instance’s life starts when the container invokes the newInstance method on the session bean class to create a new session bean instance. Next, the container injections the bean’s SessionContext, if applicable, and performs any other dependency injection as specided by metadata annotations on the bean class or by the deployment descriptor. The container then calls the PostConstruct lifecycle callback interceptor methods for the bean, if any. The container can perform the instance creation at any time — there is no direct relationship to a client’s invocation of a business method or the create method.
- The session bean instance is now ready to be delegated a business method call from any client or a call from the container to the timeout callback method.
------

Therefore after the postConstruct callback has been executed the container can perform the instance creation at any time, so still it is in creating meanwhile the method createTimer doesn't allow that state.

Hence, the workaround that you mentioned at the beginning of calling the EJB initialization of Timer after the creation of the stateless bean is a better approach.

Finally the state of the bean that is triggering the IllegalStateException has correct behavior regarding to the EJB specification.

ServletContextListener

We tried using a ServletContextListener to create ejb timers during the context initialization.

@EJB  Timer1SessionBeanLocal timer1;

public void contextInitialized(ServletContextEvent arg0) {

                timer1.createTimer();

}

It failed also.

Exception:

javax.ejb.EJBException: EJB Exception: : java.lang.NullPointerException at weblogic.ejb.container.timer.ClusteredEJBTimerManager.createTimer(ClusteredEJBTimerManager.java:76) at weblogic.ejb.container.timer.ClusteredEJBTimerManager.createTimer(ClusteredEJBTimerManager.java:95)
at weblogic.ejb.container.internal.TimerServiceImpl.createTimer(TimerServiceImpl.java:125)
at weblogic.ejb.container.internal.TimerServiceImpl.createTimer(TimerServiceImpl.java:49)

Reasons:

The weblogic team asked us to apply a patch for this problem. But we never did because of some other constraints.

timerService.getTimers()

Sometimes this function call throws a NullPointerException

weblogic.ejb.container.deployer.TimerServiceProxyImpl@3a2229fc
java.lang.NullPointerException
        at weblogic.scheduler.ejb.internal.EJBTimerManagerImpl$TimerWrapper.getListener(EJBTimerManagerImpl.java:187)
        at weblogic.ejb.container.timer.ClusteredEJBTimerManager.getTimers(ClusteredEJBTimerManager.java:123)
        at weblogic.ejb.container.timer.ClusteredEJBTimerManager.getTimers(ClusteredEJBTimerManager.java:107)
        at weblogic.ejb.container.internal.TimerServiceImpl.getTimers(TimerServiceImpl.java:158)
        at weblogic.ejb.container.deployer.TimerServiceProxyImpl.getTimers(TimerServiceProxyImpl.java:74)

Reason:
     The weblogic support team did not give a good explanation. It could be a bug in the weblogic.

Server stop/restart

As long as there is one server in the cluster still running, the timers will still function,  the application may run slower though. When all servers in the cluster stop, the timers will stop running.  When at least one of the servers in the cluster restarts again, the timers will start running.

Application stop/restart

After the EJB 3 timer application stops, the timers will stop running. When the application restarts, the timers will NOT start running. The weblogic support team said it may be a bug.

Application undeployment/redeployment

When an EJB 3 timer application is undeployed, the timers will be removed from the system, the records related to the timers for this application in the WEBLOGIC_TIMERS table will be deleted. Sometimes, some records related to this application will still exist in the ACTIVE table (it may be a bug), but it won’t affect the future deployment of the application. When the application is redeployed, a new cycle starts again. We did not use “update” for the application redeployment often, we felt it should be avoided from our experience.

Overall, we think stopping and then restarting an EJB 3 timer application should be avoided.  Use the server stop/restart or application undeployment/redeployment.

EJB 3.1 Timer

There are nice articles online regarding this new technology. Overall, the syntax is richer, the timer can be created declaratively. Since we do not have much experience on this, we do not have any to share on this.

Conclusions

Overall, EJB 3 timer provides a nice way to schedule tasks in jee environment. How to create ejb timers and use them effectively in a clustered environment is not very straightforward. We went through a sample application,  explained the implementation and configuration in  weblogic 10.3.2, and also shared the lessons we learned. We hope this will provide some useful information to users when they use EJB 3 timers  in their applications.

Acknowledgements

Big thanks to my colleague Lee Slezak at HP for constructive suggestions.

source code 

deployable ear


Other blogs from me:
Compile xsl files and store in cache to improve XSLT performance