Tuesday, December 20, 2011

Upgrading JBoss 4 to JBoss 5 with Java 5 to Java 6

The information presented here comes from an effort to upgrade a Java enterprise application to the most current versions of all of its parts; primarily to get onto Java 6. Its starting system specifications were the following:

  • JBoss 4.2.2
  • Java 1.5.0_27
  • SEAM 2.1.0
  • RichFaces 3.3.1
  • JSF 1.2
  • EJB 3.0
  • Remote EJB’s

For reasons that will be explained in detail, it was not possible to upgrade each of the system parts to their various current versions. The end result was the following:

  • JBoss 5.1.0
  • Java 1.6.0_27
  • SEAM 2.2.2
  • RichFaces 3.3.3
  • JSF 1.2
  • EJB 3.0
  • Remote EJB’s

Due to the size and scope of the enterprise application itself in combination with the incompatibilities between system components, the entire effort ended up taking me a long time. The following is a log of this endeavor with all of the errors I encountered and how they were resolved. The intention of this writing is for both a historical purpose, and to help others avoid the pitfalls of performing such an upgrade on a sufficiently large enterprise application.

Why didn’t it just work out of the box with Java 6?

JBoss 4.2.2 Web Services do not work with Java 1.6. It results in a runtime error with SOAP. This means in order to run with Java 6 we have to upgrade JBoss.

java.lang.UnsupportedOperationException: setProperty must be overridden by all subclasses of SOAPMessage

 

What about JBoss 7.0?

The configuration is so different that it in no way resembles any previous version, and it does not support Remote EJB’s.

Remote EJB invocations isn't currently available in AS 7.0. That will be implemented by AS 7.1.
(http://community.jboss.org/message/616635)

So, how about JBoss 6.1.0?

JBoss 6 supports Java 6, SEAM, RichFaces, JSF, EJB 3.0, and remote EJB’s. The question is now what versions of these will work?

What about RichFaces 4?

Yes, it works but with JBoss 6.1.0 and JSF 2.0.

http://docs.jboss.org/richfaces/latest_4_0_X/Developer_Guide/en-US/html/chap-Developer_Guide-RichFaces_overview.html

RichFaces supports the following JSF implementations and frameworks:

  • MyFaces 2 and higher
  • Seam 3 and higher
  • Mojara 2 and higher

So looks like you have to upgrade to SEAM 3

Thus far we have JBoss 6.1.0, RichFaces 4.0.0, and JSF 2.0. SEAM 3 has apparently changed everything, and is not backward compatible with a SEAM 2 configuration. I was never able to even get “Hello World” working with SEAM 3, which is to be expected since it is “not done” (http://seamframework.org/157223.lace).

While according to compatibility it is supposed to work with JBoss 6 (http://seamframework.org/Seam3/Compatibility), I again haven’t been able to get even the most simple example working.

 

Back to SEAM 2

Since SEAM 3 does not work we have no choice but to go back to SEAM 2. RichFaces requires JSF 2.0 and in its specification says it only works with SEAM 3. While I question how one could know that RichFaces 4 only works with SEAM 3 when no one to my knowledge has a working example on JBoss 6, the goal is to upgrade to Java 6. There is also some unspecified hack to get SEAM 2 working with JSF 2, but any type of hack would probably make upgrading harder in the future. Okay, back to to RichFaces 3.

 

Back to RichFaces 3, how about 3.3.3?

Sorry no, not on JBoss 6: http://docs.jboss.org/richfaces/latest_3_3_X/en/devguide/html_single/#SupportedServers

JBoss 6 isn’t even listed as a supported server, and in my experience I haven’t been able to find anyone able to get RichFaces 3 working under JSF 1.2 or 2.0 on JBoss 6. I haven’t been able to get a “Hello World” working either.

Wait, where are we now?

  • We can’t use SEAM 3 because it doesn’t work (or at least the community can’t get it to work) on JBoss 6
  • We can’t use RichFaces 4 because it requires JSF 2 and SEAM 2 requires JSF 1.2
  • RichFaces 3 isn’t supported (and won’t run apparently) on JBoss 6
  • Finally, was can’t use JBoss 6 because SEAM 2 doesn’t support it (http://seamframework.org/Seam2/GettingStarted), and of course RichFaces 3 doesn’t work with it either

JBoss 5.1.0, Hello World

On paper RichFaces 3 and SEAM 2 work on JBoss 5, and JBoss 5 can run on Java 6. Take into account though the same is true for RichFaces 4, JSF 2, SEAM 3, and Remote EJB’s on JBoss 6. While an example exists for RichFaces 3 + SEAM 2 on JBoss, it of course doesn’t work out of the box on JBoss 5.

As a starting point I used Eclipse 3.7 (Indigo), JBoss Tools 3.3.3 (http://www.jboss.org/tools/download/installation/update_3_3), and Seam 2.2.2 (http://seamframework.org/Seam2/Downloads). JBoss Tools has a “Create Seam Web Project wizard,” so it should work right out of the box, right?

First, you have to rename jboss-seam-2.2.2.jar to jboss-seam.jar in the seam-2.2.2/lib installation. This is because if you don’t the SEAM wizard won’t allow you to select where you put SEAM because it looks for the JAR of this name.

Second, RichFaces component will not work out of the box. If you try to use something like a calendar or a TabPanel you will get this message:
Resources framework is not initialised, check web.xml for Filter configuration

Not too hard; add this to the web.xml:
<filter>
<display-name>Ajax4jsf Filter</display-name>
<filter-name>ajax4jsf</filter-name>
<filter-class>org.ajax4jsf.Filter</filter-class>
</filter>
  <filter-mapping>
<filter-name>ajax4jsf</filter-name>
<url-pattern>*.seam</url-pattern>
<dispatcher>REQUEST</dispatcher>
<dispatcher>FORWARD</dispatcher>
<dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Third, there are all sorts of problems with .xhtml files:

  • layout/template.xhtml - Delete anything referencing authentication
  • layout/menu.xhtml - Delete the content of the toolBarGroup

Fourth, you have to delete the xerces and xml-apis jars out of the lib directory. These are in JBoss now.

 

EARs containing Beans and WARs with shared libraries in the root of the EAR

A detail of the project I am converting is that it contains several WAR files, in which the library dependencies reside in the root level of the EAR, instead of the WEB-INF/lib directories of the WARs.

There are a couple of interesting side-effects of this type of structure on JBoss 5:

  1. The app no longer runs in Eclipse, because the libraries don’t actually reside in the root directory of the deployment
  2. Having these dependencies in the root of the EAR interferes with the admin-console deployment. There are several ways of dealing with this, but the easiest is just to remove it from deploy. (https://issues.jboss.org/browse/JBAS-7032)

In terms of getting this configuration to work in Eclipse, you only would have to automatically copy the JAR files to the root of the deployment. The deployment assembly settings right? No. When they work, they only copy a few files. The files they choose to copy whether you use a filterset or add each one manually are seemingly random.

How about a custom builder within Eclipse to copy the files? The JBoss server in eclipse deploys to a dynamic location within .metadata\.plugins\org.jboss.ide.eclipse.as.core within your workspace, so you would need to access it through an environment variable if you were trying to create custom builder. Is there an environment variable like ${jboss_config}? There would have to be but good look figuring it out.

In order to have JBoss in a predictable place, you can modify the server settings by double-clicking on the JBoss Server in the service view, selecting the Deployment tab, and making it deploy to a custom location (http://community.jboss.org/message/564310).

Since you don’t want your Eclipse version of your EAR messing with your manually deployed one, it is best to create a second server/YOUR_PROJECT deployment directory which is a copy of the default minus deploy/admin-console. You also have to make sure to modify the JBoss runtime configuration to NOT use the default configuration, and to use the one you just created.

For example under  JBoss Runtime Configuration -> Open Launch Configuration::
--configuration=eclipse  -b localhost  -Djboss.server.base.url=file:/C:/Java/jboss-5.1.0.GA/server/

 

The story of our app...

Jars over, application.xml changed, JBoss properties-service.xml changed to use environment.properties, build changed to use Java 6 and the JBoss 5 libraries, Eclipse project modified to compile using all the new libraries, and all of the other modifications made according to “Hello World”. It should deploy just fine right?

 

Error #1: Error installing to PostClassLoader

[org.jboss.kernel.plugins.dependency.AbstractKernelController] (main) Error installing to PostClassLoader: name=vfsfile:/C:/Java/jboss-5.1.0.GA/server/myapp/deploy/MyApp.ear/ state=ClassLoader mode=Manual requiredState=PostClassLoader
org.jboss.deployers.spi.DeploymentException: Error during deploy: vfszip:/C:/Java/jboss-5.1.0.GA/server/myapp/deploy/MyApp.ear/account-application.jar/
    at org.jboss.deployers.spi.DeploymentException.rethrowAsDeploymentException(DeploymentException.java:49)
…
Caused by: java.lang.NullPointerException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.jboss.metadata.process.processor.ejb.jboss.SetDefaultLocalBusinessInterfaceProcessor.process(SetDefaultLocalBusinessInterfaceProcessor.java:114)
    at org.jboss.metadata.process.chain.ejb.jboss.JBossMetaDataProcessorChain.process(JBossMetaDataProcessorChain.java:115)
    at org.jboss.ejb3.deployers.Ejb3MetadataProcessingDeployer.deploy(Ejb3MetadataProcessingDeployer.java:115)
    at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)

Possibilities:

This stack trace is pretty useless, and the only matches for any part of it on the internet are to those two JIRA issues. Resorting to just a “how to go from JBoss 4 to 5” search:

Nothing works, now what?

An issue with all these guides though is that they are for applications most likely smaller than the one we are upgrading. Sticking will all the advice still doesn’t have an effect on this startup error. The next resort is to break our application, which currently sits as one big Java project that can only be run by ant build and deploy, into its component pieces and get them working one at a time. Starting with the EJB that contains all the Java code that is causing this error.

Our project’s directory structure is the following:

  • Application
    • src (EJB)
    • ear
    • WebEar
      • Application WAR #1
      • Application WAR #2
      • Application WAR #3
      • Application WAR #4
    • lib

I have experimented with running this configuration out of Eclipse, and while it is possible the configuration is extremely complicated. I however also like to avoid developer setup documents which are 10 pages long, so sometimes the path of least resistance is the best way to go.

The advantage of being able to run out of Eclipse is that you don’t have to Ant build, deploy it, and startup JBoss every time you want to see a change. A different project structure also lets things be broken apart and tested more easily, because they don’t rely on you having to modify an already complicated Ant build.

The idea is that like “Hello World,” a project that we can get running easily in Eclipse should be in this project directory structure:

  • Application EAR (Enterprise Application Project)
    • lib
  • Application Bean (EJB Project with no client)
  • Application WAR #1 (Dynamic Web Project)
  • Application WAR #2 (Dynamic Web Project)
  • Application WAR #3 (Dynamic Web Project)
  • Application WAR #4 (Dynamic Web Project)

The reliance on the Eclipse project wizards results in projects that run in Eclipse out-of-the-box.

Checklist

  • EAR deploys with Bean
  • EAR deploys with Bean and application.xml contains all of the needed libraries
  • EAR deploys with Bean, libraries, and data sources
  • Error: EAR deploys with Bean, libraries, data sources, and messaging config

Error #2: Queue[/queue/myapp/bulkMessageQueue

ERROR [ExceptionUtil] Queue[/queue/myapp/bulkMessageQueue, name=myapp/bulkMessageQueue] startService
javax.naming.NameNotFoundException: myapp not bound

The issue is that JMS configuration has changed between 4 and 5: http://javabeanz.wordpress.com/2009/06/05/configuring-jms-in-jboss-5/

For myapp-jms-service.xml:

JBoss 4:

<mbean code="org.jboss.mq.server.jmx.Queue"
    name="jboss.mq.destination:service=Queue,name=myapp/bulkMessageQueue">
        <depends optional-attribute-name="DestinationManager">
            jboss.mq:service=DestinationManager
        </depends>
</mbean>

JBoss 5:

<mbean code="org.jboss.jms.server.destination.QueueService"
     name="jboss.messaging.destination:service=Queue,name=myapp/bulkMessageQueue"
     xmbean-dd="xmdesc/Queue-xmbean.xml">
     <depends optional-attribute-name="ServerPeer">
        jboss.messaging:service=ServerPeer
     </depends>
     <depends>jboss.mq:service=DestinationManager</depends>
     <attribute name="JNDIName">queue/myapp/bulkMessageQueue</attribute>
</mbean>

Back to the checklist:

  • EAR deploys with Bean, libraries, data sources, messaging config, and mail config
    … and the Seam interceptors in the ejb-jar.xml
  • ERROR: … add all the source code for the bean

Back to ERROR #1: Error installing to PostClassLoader

org.jboss.kernel.plugins.dependency.AbstractKernelController] (main) Error installing to PostClassLoader: name=vfsfile:/C:/Java/jboss-5.1.0.GA/server/myapp/deploy/MyApp.ear/ state=ClassLoader mode=Manual requiredState=PostClassLoader

We now know that this error has something to to with one or more of the following:

  • Java source - Can’t really remove this
  • jboss.xml - Removing this gets rid of the error
  • ehcache.xml
  • persistence-container.xml
  • The WSDL files

Ding, ding, ding! Something is wrong with the jboss.xml

jboss.xml looks like this:

<jboss>
    <enterprise-beans>
        <!-- Actions -->
        <session>
            <ejb-name>BasketActionImpl</ejb-name>
            <jndi-name></jndi-name>
            <clustered>false</clustered>
        </session>
        <session>
            <ejb-name>ContractPricingActionImpl</ejb-name>
            <clustered>false</clustered>
        </session>
...

The presence of one or more of those beans is what is causing the error. A cursory look has also revealed that several of the beans here don’t exist. I eventually found out that the reason for the failures was related to ejb-name classes that don’t exist. After removing classes (one at a time) jboss was able to launch with this configuration. Note though that removing the clustering settings causes error #12.

 

ERROR #3: Can't find a persistence unit

org.jboss.deployers.spi.DeploymentException: Error deploying MyEJB.jar: Exception while processing container metadata for EJB: DBServiceImpl in unit: MyEJB.jar
…
Caused by: java.lang.IllegalArgumentException: Can't find a persistence unit named 'MyPu' in AbstractVFSDeploymentContext@31521104{vfsfile:/C:/Java/jboss-5.1.0.GA/server/eclipse/deploy/MyApp.ear/MyEJBjar/}

There is obviously a problem with the persistence-container.xml, as it is the one that defines the datasource which cannot be found. According to the thread at http://community.jboss.org/thread/146523, there are a couple different ways to deal with persistence.xml. Note that is is always referred to as “persistence.xml”. If you change the name of the this XML file to persistence.xml, it works. This is an indication that JBoss 5 wants your persistence unit to be defined in persistence.xml by default.

Back to the checklist:

  • ...and WAR #1 with a relatively empty configuration
  • ERROR: ...and WAR #1 with RichFaces and SEAM in the configuration from Hello World

Error #4: NoClassDefFoundError: org/jboss/resteasy/specimpl/UriInfoImpl

java.lang.ClassNotFoundException: javax.ws.rs.ext.Provider
java.lang.NoClassDefFoundError: org/jboss/resteasy/specimpl/UriInfoImpl

This is because of the inclusion of jboss-seam-resteasy-2.2.2.jar, which does not include the REST Easy dependencies. This can just be removed from the application.xml.

 

Error #5: Server cycles between deploying and undeploying

2011-10-17 12:14:03,137 INFO  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (HDScanner) deploy, ctxPath=/account
2011-10-17 12:14:03,199 INFO  [org.apache.catalina.startup.ContextConfig] (HDScanner) WARNING: Security role name NONE used in an <auth-constraint> without being defined in a <security-role>
2011-10-17 12:14:03,215 INFO  [javax.enterprise.resource.webcontainer.jsf.config] (HDScanner) Initializing Mojarra (1.2_12-b01-FCS) for context '/account'
2011-10-17 12:14:05,324 INFO  [javax.servlet.ServletContextListener] (HDScanner) Welcome to Seam 2.2.2.Final
2011-10-17 12:14:09,434 WARN  [org.jboss.seam.security.permission.PersistentPermissionResolver] (HDScanner) no permission store available - please install a PermissionStore with the name 'org.jboss.seam.security.jpaPermissionStore' if persistent permissions are required.
2011-10-17 12:14:09,621 INFO  [org.quartz.simpl.SimpleThreadPool] (HDScanner) Job execution threads will use class loader of thread: HDScanner
2011-10-17 12:14:09,637 INFO  [org.quartz.core.QuartzScheduler] (HDScanner) Quartz Scheduler v.1.5.2 created.
2011-10-17 12:14:09,637 INFO  [org.quartz.simpl.RAMJobStore] (HDScanner) RAMJobStore initialized.
2011-10-17 12:14:09,637 INFO  [org.quartz.impl.StdSchedulerFactory] (HDScanner) Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
2011-10-17 12:14:09,637 INFO  [org.quartz.impl.StdSchedulerFactory] (HDScanner) Quartz scheduler version: 1.5.2
2011-10-17 12:14:09,637 INFO  [org.quartz.core.QuartzScheduler] (HDScanner) Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
2011-10-17 12:14:09,902 WARN  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (HDScanner) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jboss/cache/pojo/jmx/PojoCacheJmxWrapperMBean
2011-10-17 12:14:20,871 INFO  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (HDScanner) undeploy, ctxPath=/account

I encountered this issue earlier and just blamed it on Eclipse, but closer attention to the logs reveals there is a missing class. This JAR can be downloaded here: http://www.java2s.com/Code/Jar/JKL/Downloadpojocachejar.htm

Followed by:
2011-10-17 13:15:05,809 WARN  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jboss/cache/jmx/LifeCycle

Apparently life has moved on past JBoss Cache, http://community.jboss.org/wiki/JBossCache, but you can get jbosscache here: http://www.java2s.com/Code/Jar/JKL/Downloadjbosscachejar.htm

Followed by:
2011-10-17 13:23:37,277 WARN  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jboss/cache/CacheManager

The JAR can be downloaded here: http://www.java2s.com/Code/Jar/JKL/Downloadjbosscachecorejar.htm

Followed By:
2011-10-17 14:14:10,356 WARN  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jgroups/blocks/MethodCall

jgroups.jar, which can be obtained in the JBoss Cache release.

Followed by:
2011-10-17 13:34:08,809 WARN  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. ClusteringNotSupportedException: Could not access CacheManager for JBossWeb clustering

At this point I am baffled, as there are not any other obvious signs of why JBoss decided it doesn’t like this context and undeploys it. The message “Could not access CacheManager for JBossWeb clustering” on Google only points to the source code from where it came, so I am either way off or this is not related.

Upon further research, these messages have nothing to do with the undeploy. There is a majorly annoying bug in all of JBoss 5: http://stackoverflow.com/questions/1451614/jboss-deploy-undeploy-loop-caused-by-hdscanner-in-a-exploded-ear

You pretty much can’t have anything but .xml files in your WAR WEB-INF directory. Removing a .jpg from here resolved this issue.

 

Error #6: SEAM Actions don’t resolve

2011-10-17 15:08:59,774 WARN  [org.jboss.ejb3.interceptors.aop.InterceptorsFactory] (http-localhost%2F127.0.0.1-8080-1) EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container
2011-10-17 15:08:59,805 WARN  [org.jboss.ejb3.interceptors.aop.InterceptorsFactory] (http-localhost%2F127.0.0.1-8080-1) EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container
2011-10-17 15:09:13,555 WARN  [org.jboss.seam.security.jaas.SeamLoginModule] (http-localhost%2F127.0.0.1-8080-2) Error invoking login method
javax.el.PropertyNotFoundException: Target Unreachable, identifier 'loginAction' resolved to null

Apparently there is a problem with the SEAM security configuration, which is what controls login. This is to be expected since security changed in SEAM 2.2.2: http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/5/html/Seam_Reference_Guide/Migration20.Changes.html

There is also of course different XSD’s now available, which require that you change all the -2.1.xsd’s to 2.2.xsd’s.

The following post indicates that the problem may be that the class inside the JAR within the EAR is not being picked up by SEAM, because there is no seam.properties files in the JAR: http://stackoverflow.com/questions/3978976/can-the-seam-authenticator-reside-in-the-ear-file-lib-folder

This was the problem. If you have beans in a JAR that JAR must define an empty seam.properties file in order for it to be scanned for beans.

 

Error #7: javax.naming.NameAlreadyBoundException: classPathEntries

Apparently there is an issue with the hibernate-core jar existing in both the ears in the deployment, this was resolved by deleting deleting the second ear. Eventually the second EAR was made to not include this JAR.
Also had the same problem with javassist

Error #8: Unit Tests no longer run out of Eclipse

The problem is running unit tests with a classpath that is too long in Eclipse.

You have to edit the classpath of the target to not include external dependencies

 

Error #9: Some pages are 404 not found

This was a really annoying problem that plagued me for a long time. Buttons that used to correctly direct to htm pages were no longer working. Upon closer inspection the link that works is going to pageC.htm while the one that doesn’t is going to pageB.xhtml. Something is not rewriting xhtml to htm in the page rendering.

The problem the entire time has been these pages were named “xtml” instead of “xhtml” and I didn’t see it. Apparently the previous version of SEAM, JBoss, Java, or something else handled xtml even though it is not a part of the configuration.

 

Error #10: java.lang.InstantiationException: org.apache.log4j.Logger

javax.ejb.EJBException: java.lang.RuntimeException: org.jboss.serial.exception.SerializationException: Could not create instance of org.apache.log4j.Logger - org.apache.log4j.Logger
...
caused by: java.lang.InstantiationException: org.apache.log4j.Logger

Google only provided two solutions: make the logger static or declare it as @Transient

Doing both of these didn’t get rid of the error.

There was a warning about a @Destory method in a class that was using a log4j logger, so I changed it to be seam injected. This too didn’t work, even when marked with @Transient.

As it turns out the way to fix this is simple, only use static transient Loggers. In my application this means I had to do this 890 times. A result of this problem is also a memory leak, because the beans can’t be destroyed.

 

Error #11: EJB Timer throws a null pointer exception

2011-10-28 09:19:54,584 ERROR [org.jboss.ejb.txtimer.TimerImpl] (EJB-Timer-1319811479419[target=jboss.j2ee:ear=Irio.ear,jar=jboss-seam-2.2.2.jar,name=TimerServiceDispatcher,service=EJB3]) Error invoking ejbTimeout
javax.ejb.EJBException: java.lang.RuntimeException: java.lang.NullPointerException

It doesn’t prevent startup and corrects itself, but slows the start up process down. There doesn’t appear to be anyway to get rid of getting this exception at least once on start up.

However, it should be noted that if you start getting these errors several times during startup it is an indication that JBoss will fail to startup eventually. The only way to prevent this is to delete the tmp, work, and data directories every time you restart a JBoss instance.

 

Error #12: Caused by: java.lang.NoClassDefFoundError: org/jgroups/Channel

The solution is to add the jgroups jars to lib/endorsed for whatever server you are running.

Reference: http://community.jboss.org/message/440606

But then you get a stack:

2011-11-03 09:57:33,359 ERROR [org.jboss.ejb3.cache.simple.SimpleStatefulCache.SMSOptionsActionImpl] (SFSB Passivation Thread - jboss.j2ee:ear=Irio.ear,jar=IrioAccountApplication.jar,name=SMSOptionsActionImpl,service=EJB3) problem passivation thread
javax.ejb.EJBException: Could not passivate; failed to save state

Reference: http://seamframework.org/Documentation/HandlingPrePassivateCleanupInSFSBs

Reading this article lead me to take a look at clustering again, which has to do with Error #1. As is turned out setting the troublesome class to clustered = false in the jboss.xml seems to have resolved this issue.

I again had to come back to this, and found that further down in the stacktrace there is a message about a missing class.

Long story short, I had to add the following jars to lib/endorsed:

  • mail.jar
  • log4j.jar
  • commons-logging.jar
  • jbosscache.jar
  • jgroups-all.jar
  • pojocache.jar
  • jbosscache-core.jar

The error following the passivation error, was java.lang.NoClassDefFoundError: org/jboss/cache/pojo/jmx/PojoCacheJmxWrapper

When I looked in the logs prior to all of this and during startup, I found that there was a warning message stating the clustering was disabled because orjava.lang.NoClassDefFoundError: org/jboss/cache/pojo/jmx/PojoCacheJmxWrapper. So I figured if I got rid of this startup error that it would get rid of the passivation error. The problem though is that they only way to get this error is to do something involving the bean, and then wait several hours.

 

Error #13: Database connection hangs forever (using MS SQL)

The reason for this has to do with a problem between Java 1.6.0_29 and the MS SQL JDBC driver. The only solution is to use Java 1.6.0_27, which is why that is the version of Java listed in the new spec even though it is not the most recent version of Java 6.

Reference: http://stackoverflow.com/questions/7841411/driver-getconnection-hangs-using-sqlserver-driver-and-java-1-6-0-29

 

Error #14: JBOSS_HOME and JAVA_HOME environment variables do to propagate into scheduled tasks

http://community.installshield.com/showthread.php?t=147931 indicates that the only way Scheduled Tasks can pick up changes to environment variables is to reboot the machine. However, in my case I was not able to reboot the machine so I had to find a workaround.

To do it without having to reboot:

  1. Login as the non-running user
  2. Disable all Scheduled Tasks
  3. Change the environment variable
  4. Start the Task Manager and select to show all processes from all users
  5. In the Task Manager, kill all tasks being run by the Scheduled Task running user (taskeng.exe)
  6. Log Off
  7. Log on as the non-running user
  8. Enable all Scheduled Tasks

ERROR #15: java.lang.OutOfMemoryError: unable to create new native thread

This issue has to do with the following things:

  • The amount of memory allocated with Xmx. This is in your conf if using the service wrapper.
  • The amount of memory allocated with Xms. This is in your conf if using the service wrapper.
  • The maximum number of AJP threads allowed in jbossweb.sar/server.xml

The key is finding the balance between these values, which is dependent on Java version, what your application is doing, and how much memory is available on your machine.

Reference: http://blog.egilh.com/2006/06/2811aspx.html

 

Error #16: Failed to parse source: {http://java.sun.com/xml/ns/javaee}virtual-host

org.jboss.xb.binding.JBossXBException: Failed to parse source: {http://java.sun.com/xml/ns/javaee}virtual-host cannot appear in this position. Expected content of {http://java.sun.com/xml/ns/javaee}web is unordered_sequence: {http://java.sun.com/xml/ns/javaee}context-root? {http://java.sun.com/xml/ns/javaee}web-uri?

The following is no longer valid, as virtual-host isn’t allowed here.
<module>
      <web>
         <web-uri>foo.war</web-uri>
         <context-root>/foo</context-root>
         <virtual-host>myhost</virtual-host>
      </web>
   </module>

Remove it because it should be defined in jboss-web.xml anyways.

 

Error #17: javax.ejb.EJBNoSuchObjectException: Could not find Stateful bean: 41z4l26-wb41k6-ev4j007b-1-ev4jirql-bw

I have been seeing this error in the logs quite often and in the previous version of JBoss, being 4.2.2. It didn’t appear to be critical so I jut let it be like the hundreds of other errors. I however stumbled across and article explaining the cause for this: http://community.jboss.org/wiki/JbossTimeoutSettingForSeam

To summarize, you get this problem with stateful session bean timeouts do not match the session timeout. This is great considering that the default session timeout (web.xml) is 30 minutes while the default stateful session bean timeout is 5 minutes (ejb3-intercepters-aop.xml).

 

Warning #1: EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container

This is something that can be ignored entirely, and the developers even recommend that you do. Every time you access a bean you will get this warning, so it floods the log.

Reference: http://stackoverflow.com/questions/491007/jboss-what-does-the-warning-ejbthree-1246-from-the-interceptorregistry-mean

Posted by at 4 comments:

Tuesday, August 23, 2011

Using DBUnit through Ant with a large MS SQL Database

The following describes my experiences with using DBUnit through Ant with a large MS SQL database, and all of the adventures I had. The intention is to use DBUnit though Ant to export a database as an XML file, to clear the database, and to re-populate it using that XML file.

Why?

Automated acceptance testing. You start with an XML backup of what you want your base system to contain, refresh your database with that backup, run your suite of automated acceptance tests, and then restore the XML backup again. That way your tests can do things specific to the application like create users, invoices, etc, and otherwise mess around with every conceivable operation. When you are done you can restore everything to the way it was before.

Running it though Ant lets you easily integrate this refresh into other automation tools such as Jenkins (Hudson), TeamCity, and so on.

Did you say large?

Yes I did, but in this context I am using it to refer to a database with a large number of tables, not a large number of records. A large number of tables is a catch-all for several circumstances which you cause you trouble, such as with tables which have keywords for names, foreign key dependencies between tables, tables with timestamp columns, and some more items I will get address.

But I want a lot of records!

There are a couple of problems with using a lot of records with DBUnit, the first of which is that if you are exporting and importing you will have to deal with very long save and load times. As a general rule you probably are aiming for an export XML file size of under 3 MB, which is around 2,000 records depending on what it is your tables. The second problem has to do with memory, but I have a workaround for that as well.

DBUnit with Ant

So you want to use Ant? You are then going to have to write your own task by extending the DBUnit Ant Task. This is because there are options that need to be set [at least to my knowledge] that cannot be set in the DBUnit Ant Task that is provided to you.

To start out you will need to extend three things:

  1. The DbUnitTask, which is the <dbunit> tag. This allows you to define your own Export and Operation tags.
  2. The Export, which is the <export> tag. This is for changing how the export works.
  3. The Operation, which is the <operation> tag. This is for changing how the insert works.

MSDBUnitTask.java

public class MSDBUnitTask extends DbUnitTask {

    @SuppressWarnings("unchecked")
    public void addMSExport(MSExport export) {
        this.getSteps().add(export);
    }
    @SuppressWarnings("unchecked")
    public void addMSOperation(MSOperation operation) {
        this.getSteps().add(operation);
    }
}

MSExport.java

public class MSExport extends Export   {

    private String tableNames;
    @Override
    public void execute(IDatabaseConnection arg0) throws DatabaseUnitException {
        // code goes here
    }

    public String getTableNames() {
        return tableNames;
    }

    public void setTableNames(String tableNames) {
        this.tableNames = tableNames;
    }

}

MSOperation.java

public class MSOperation extends Operation   {

        @Override
    public void execute(IDatabaseConnection arg0) throws DatabaseUnitException {
        // code goes here
    }

}

Including your custom task in Ant

This assumes that the Ant classes from above are compiled into a JAR and present in the ${lib.dir} along with the dbunit.jar

<path id="classpath">

    <fileset dir="${lib.dir}">
        <include name="*.jar" />
    </fileset>   
</path>

<taskdef name="msdbunit" classname="com.foo.ant.dbunit.MSDBUnitTask" classpathref="classpath" />

 

Adventures in Exporting

The Java code here goes in the execute method of MSExport.

This is where the majority of problems start. As it turns out you can export with ease, but can end up with XML that had invalid characters and some other interesting things that cannot be imported.

Let’s start with the result we want (and need): the XML to export the database:

<target name="db-export">

    <msdbunit
        driver="${db.driver}" 
        url="${db.url}" 
        userid="${db.username}" 
        password="${db.password}">

        <dbconfig>
            <property name="escapePattern" value="[?]" /> 
            <property name="datatypeFactory" value="org.dbunit.ext.mssql.MsSqlDataTypeFactory" /> 
        </dbconfig>
        <msexport format="xml" tableNames="${insert.table.names}" dest="full_export.xml" />
    </msdbunit>

</target>

The msdbunit and msexport tags were explained earlier, and the base Java code for creating them was given.

Issue #1: Export Table Order

You must specify the order of tables to do the export in insertion order. This is done by passing in a comma separated list of tables name from ${insert.table.names} into the <msexport> tag as the tableNames attribute.

The reason for this order is that you must take into account FK dependencies when both inserting and deleting. For example if Foo.A is a FK to Bar.A, then you can’t start out by inserting records into Foo; you have to insert into Bar first. From the deletion perspective it is the opposite: You can’t delete from Bar since Foo can reference it and cause the delete to fail.

The MSExport task already has tableNames defined, so in order to get the result as a list of tables all that needs to be done is this:

String[] tableNamesArray = tableNames.split(",");

ITableFilter filter = new SequenceTableFilter(tableNamesArray);

IDataSet dataset = new FilteredDataSet(filter, arg0.createDataSet());

The second part of this is do use this array to define the table sequence. You must do this manually because even though it can be done automatically, with 100+ tables it will take minutes as opposed to a fraction of a second.

Issue #2: Columns that are of type TIMESTAMP or default DATETIME

There is an issue if you export a column of the TIMESTAMP type or of the defaulted DATETIME, because when you try and import it you will get an error regarding not being able to insert a value into the column.

The exact error you get is the following:

Cannot insert an explicit value into a timestamp column

The solution is not to export these columns, which requires the use of a column filter. This was the primary reason for abandoning the default Ant task, as I couldn’t figure out a good way to do a large column filtering.

List<ITable> updatedTables = new ArrayList<ITable>();

for (String tableName : tableNamesArray) {
                ITable table = dataset.getTable(tableName);
                List<Column> excludedColumns = new ArrayList<Column>();
                Column[] columns = table.getTableMetaData().getColumns();
                //For each column...
                for (Column column : columns) {
                    //TIMESTAMP columns have to be ignored in MSSQL because they cannot have an explicit insert
                    //Unfortunately DATETIME in MSSQL also shows here as a TIMESTAMP so the only way to
                    //determine defaulted DATETIME or TIMESTAMP is by looking for defaults and the SQL Type name
                    if ( (column.getDefaultValue() != null && column.getDataType() == DataType.TIMESTAMP) ||
                            column.getSqlTypeName().equals("timestamp")) {
                        excludedColumns.add(column);
                    }
                }
                //convert the list of excluded columns to an array
                Column[] excluded =  excludedColumns.toArray(new Column[excludedColumns.size()]);
                //create a new ITable that excludes the filtered columns
                ITable filteredTable = DefaultColumnFilter.excludedColumnsTable(table,excluded);
                //keep track of the modified table
                updatedTables.add(filteredTable);               
            }
            //Take all of the modified table and create a new dataset
            ITable[] updateTableArray =  updatedTables.toArray(new ITable[updatedTables.size()]);
            CompositeDataSet compositeDataSet = new CompositeDataSet(updateTableArray);

This code iterates over all of the tables, looks for datetime and timestamp columns, excludes them from that table, and adds them to a new list of tables with the appropriate columns excluded. It then constructs a new composite dataset containing all of the new tables and columns.

Issue #3: Exporting your customizations

When you extend the Export task, you have to then define how to export.

//Get the format and file destination
String format = this.getFormat();
File dest = this.getDest();
//Output based on the format
if (format.equalsIgnoreCase(FORMAT_CSV)) {
    CsvDataSetWriter.write(compositeDataSet, dest);
else if (format.equalsIgnoreCase(FORMAT_FLAT)) {
    FlatXmlDataSet.write(compositeDataSet, new FileOutputStream(dest));
} else if (format.equalsIgnoreCase(FORMAT_XML)) {
    XmlDataSet.write(compositeDataSet, new FileOutputStream(dest));
} else if (format.equalsIgnoreCase(FORMAT_DTD)) {
    FlatDtdDataSet.write(dataset, new FileOutputStream(dest));
} else {
    throw new DatabaseUnitException(format+" is not a recognized format.");
}

Issue #4: Exporting Tables that have keywords for names

For example User is a keyword, but you can have a table of this name in MS SQL and access it using [User]. In order to deal with this you want to put all table names within [].

This can be configured in the Ant XML using the escapePattern property of dbconfig:

<target name="db-export">

    <msdbunit
        driver="${db.driver}" 
        url="${db.url}" 
        userid="${db.username}" 
        password="${db.password}">

        <dbconfig>
            <property name="escapePattern" value="[?]" /> 
            <property name="datatypeFactory" value="org.dbunit.ext.mssql.MsSqlDataTypeFactory" /> 
        </dbconfig>
        <msexport format="xml" tableNames="${insert.table.names}" dest="full_export.xml" />
    </msdbunit>

</target>

Issue #5: Data Type Factory

In order to be able to insert records into a MS SQL database using DBUnit, you have to specify the data factory to use as a part of the export. If you don’t any time you try and import a result containing a primary key you will get the following error:

Cannot insert explicit value for identity column in table

This issue can also be resolved in the default Ant using the dataTypeFactory property in the dbconfig:

<target name="db-export">

    <msdbunit
        driver="${db.driver}" 
        url="${db.url}" 
        userid="${db.username}" 
        password="${db.password}">

        <dbconfig>
            <property name="escapePattern" value="[?]" /> 
            <property name="datatypeFactory" value="org.dbunit.ext.mssql.MsSqlDataTypeFactory" /> 
        </dbconfig>
        <msexport format="xml" tableNames="${insert.table.names}" dest="full_export.xml" />
    </msdbunit>

</target>

Issue #6: Which table failed to export?

This issue is that when you are exporting your database and something goes wrong, it is not specified which table was the problem. You just get an error message such as “Cannot insert explicit value for identity column in table” and are left to figure out which table may have caused it.

Since we have our own Ant task that iterates over tables, this is pretty easy to do: put a System.out.println with the table name at the start of the loop. That was if something breaks, you know which table caused it.

Deleting from the Database

If you exported correctly this isn’t a problem, and doesn’t require any custom work beyond the escapePattern and data type factory. It should be noted that before inserting into your database you want to clear it using your export file first. The CLEAN_INSERT operation is supposed to do this on insert for you, but in my experience I have never been able to get it work on a large database. It ends up throwing constraint violations, either because of the database, a bug, or my misuse of the operation.

<target name="db-delete">

    <msdbunit
        driver="${db.driver}" 
        url="${db.url}" 
        userid="${db.username}" 
        password="${db.password}">

        <dbconfig>
            <property name="escapePattern" value="[?]" /> 
            <property name="datatypeFactory" value="org.dbunit.ext.mssql.MsSqlDataTypeFactory" /> 
        </dbconfig>
        <operation type=”DELETE_ALL” format="xml" src="full_export.xml" />
    </msdbunit>

</target>

Populating the Database

The Java code here goes in the execute method of MSOperation.

The same as with deleting from the database, if you correctly exported the dataset you should not have any problems. There is one feature though that is nice to have, which is the ability in the case of an error to know which table cause the error.

        //Only perform an operation for every table if doing a clean insert for MS SQL
        if (this.getType().equalsIgnoreCase("MSSQL_CLEAN_INSERT")) {
            try {
                FileReader reader = new FileReader(this.getSrc());
                String format = this.getFormat();
                IDataSet dataset = null;
                if (format.equalsIgnoreCase(FORMAT_XML)) {
                    dataset = new XmlDataSet(reader);
                } else {
                    throw new DatabaseUnitException("xml is the only supported format.");
                }
                String[] tableNames = dataset.getTableNames();
                //For each table, create a new dataset and execute the operation because:
                for (int i = 0; i < tableNames.length; i++) {
                    String tableName = tableNames[i];
                    System.out.println("Populating table "+tableName);
                    ITable table = dataset.getTable(tableName);
                    ITable[] iTables = new ITable[1];
                    iTables[0] = table;
                    CompositeDataSet composite = new CompositeDataSet(iTables);
                    this.getDbOperation().execute(connection, composite);
                }
            } catch (Exception e) {
                e.printStackTrace();
                throw new DatabaseUnitException(e);
            }
        } else {
            super.execute(connection);
        }

    }

Beyond that all you need is the following:

  • MSSQL_CLEAN_INSERT
  • The MS SQL Data Type Factory
  • The escapePattern for tables with keywords for names

<target name="db-populate">

    <msdbunit
        driver="${db.driver}" 
        url="${db.url}" 
        userid="${db.username}" 
        password="${db.password}">

        <dbconfig>
            <property name="escapePattern" value="[?]" /> 
            <property name="datatypeFactory" value="org.dbunit.ext.mssql.MsSqlDataTypeFactory" /> 
        </dbconfig>
        <msoperation type=”MSSQL_CLEAN_INSERT” format="xml" src="full_export.xml" />
    </msdbunit>

</target>

Where did I get my information?

The internet combined with trial and error. Here are the sites I used for information:

Posted by at No comments:
Labels:

Tuesday, August 16, 2011

SQL query performance using a select on a large record set with Hibernate

This title is partly a question because I am unsure that my solution is the best. I have come to the following solution based on trial and error, and settling on what seemed to have worked.

The Database

The database is MS SQL 2008 Enterprise, we are using the correct JDBC driver, and are using Hibernate 3.4.0.GA. The database is also located on the same physical machine as the application server.

The Table

There is only a single table that is the target of the problem, which has the following structure:

Foo

  • ID (UUID Primary Key)
  • A (int, indexed)
  • B (datetime, indexed)
  • C (varchar, indexed)
  • D (int, not indexed)

In this table there are slightly over 2.5 million records, and I am trying to select less than 10 results.

The Query

This is the query that was slow, taking over 5 minutes to complete (in the SQL Variant). Take into account though that the different columns were used under different circumstances, making the query dynamic. The more columns used, the slower the query. In the case of using every column it looked like this:

SQL:

SELECT ID, A, B, C, D

FROM FOO

WHERE A = 1 AND B >= ‘2011-08-12’ AND C = ‘BAR’ AND D = 3

Hibernate:

Query query = em.createQuery("from Foo where a=:a and b >= :b and c = :c and d = :d”);

query.setParameter(“a”, 1);

query.setParameter(“b”, date);

query.setParameter(“c”, “BAR”);

query.setParameter(“d”, 3);

List<Foo> list = query.getResultList();

The Obvious

Column D isn’t indexed. Column D isn’t also used all of the time, and removing it from the query only cuts the 5 minute time by 30 seconds. There is also the preference that I do not change the existing database unless absolutely the only option.

There are also probably better ways to deal with the datetime column type, for example see http://diegworld.blogspot.com/2010/09/sql-server-performance-querying-by-date.html. However, if I remove this column from the query it doesn’t change the amount of time it takes.

The SQL Solution

Eventually I just started playing around with the SQL variant of the query to see if I could improve the performance. I started with running select statements with each individual column and found that they all worked in a second, for example:

SELECT ID, A, B, C, D, FROM FOO WHERE A = 1

SELECT ID, A, B, C, D, FROM FOO WHERE B >= ‘2011-08-12’

SELECT ID, A, B, C, D, FROM FOO WHERE C = ‘BAR’

SELECT ID, A, B, C, D, FROM FOO WHERE D = 3

At this point I realized that the problem has to do with all of the restrictions I added in the where clause, but changing the order of the “ands” didn’t improve performance either. With this information I then tried building the query so that it would incorporate sub queries. The idea being that if I have select with A I have set A, in which I can select with B from that and have Set AB, select C from that and have Set ABC, and select D from that and have Set ABCD. The resulting Set of ABCD would contain only records which had the A, B, C, and D criteria.

The resulting SQL that I came up with was the following:

SELECT ID, A, B, C, D FROM FOO fooA WHERE fooA.ID IN (

SELECT ID FROM FOO fooB WHERE fooB.ID IN (

    SELECT ID FROM FOO fooC WHERE fooC.ID IN (

          SELECT ID FROM FOO fooD WHERE D = 3

    ) AND fooC.C = ‘BAR’

) AND fooB.B >= ‘2011-08-12’

) AND fooA.A = 1

This query took less than a second to run, as opposed to the previous variant that was taking more than 5 minutes.

My question is though, is this the best thing to do from both a SQL and MSSQL perspective?

The Hibernate Implementation

Hibernate has a lot of flexibility in terms of how you can do sub queries, but according to my research on the internet the preferred means for doing this is to use Criteria and DetachedCriteria. Here are some places which I used to come to this conclusion:

The way this works in Hibernate is as follows:

Session session = (Session) getEntityManager().getDelegate();

Criteria crit = session.createCriteria(Foo.class);

// First subquery to get A

DetachedCriteria query1 = DetachedCriteria.forClass(Foo.class);

query1.add(Restrictions.eq("a", 1));

query1.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query1));

// Second subquery to get B

DetachedCriteria query2 = DetachedCriteria.forClass(Foo.class);

query2.add(Restrictions.ge("b", date));

query2.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query2));

// Third subquery to get C

DetachedCriteria query3 = DetachedCriteria.forClass(Foo.class);

query3.add(Restrictions.eq("c", “BAR”));

query3.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query3));

// Fourth subquery to get D

DetachedCriteria query4 = DetachedCriteria.forClass(Foo.class);

query4.add(Restrictions.eq("d", 3));

query4.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query4));

List<Foo> results = crit.list();

Each subquery reads as “get the id’s from Foo where the restriction holds true”, and the criteria represents the combined result of all of the added subqueries.

This isn’t the exact equivalent of the previously specified SQL though. When looking at the HQL debug output it ends up being the following:

SELECT ID, A, B, C, D

FROM Foo

WHERE ID in (SELECT ID FROM Foo WHERE A = 1)

AND ID in (SELECT ID FROM Foo WHERE B >= ‘2011-08-12’)

AND ID in (SELECT ID FROM Foo WHERE C = ‘BAR’)

AND ID in (SELECT ID FROM Foo WHERE D = 3)

This is better than what was there, but it still isn’t as efficient as it could be. In this code we are selecting 4 different times across the entire Foo record set. In the SQL I am trying to write each time we select it is from the previous subset. For example we are looking for condition B inside the A result set,which is smaller than all of the Foo records.

As it turns out, it isn’t that hard. You just add your subqueries to other subqueries like this:

Session session = (Session) getEntityManager().getDelegate();

Criteria crit = session.createCriteria(Foo.class);

 

// First subquery to get A

DetachedCriteria query1 = DetachedCriteria.forClass(Foo.class);

query1.add(Restrictions.eq("a", 1));

query1.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query1));

 

// Second subquery to get B

DetachedCriteria query2 = DetachedCriteria.forClass(Foo.class);

query2.add(Restrictions.ge("b", date));

query2.setProjection(Projections.property("id"));

query1.add(Subqueries.propertyIn("id", query2));

 

// Third subquery to get C

DetachedCriteria query3 = DetachedCriteria.forClass(Foo.class);

query3.add(Restrictions.eq("c", “BAR”));

query3.setProjection(Projections.property("id"));

query2.add(Subqueries.propertyIn("id", query3));

 

// Fourth subquery to get D

DetachedCriteria query4 = DetachedCriteria.forClass(Foo.class);

query4.add(Restrictions.eq("d", 3));

query4.setProjection(Projections.property("id"));

query3.add(Subqueries.propertyIn("id", query4));

 

List<Foo> results = crit.list();

This hibernate code results in the desired SQL:

SELECT ID, A, B, C, D FROM FOO fooA WHERE fooA.ID IN (

SELECT ID FROM FOO fooB WHERE fooB.ID IN (

    SELECT ID FROM FOO fooC WHERE fooC.ID IN (

          SELECT ID FROM FOO fooD WHERE D = 3

    ) AND fooC.C = ‘BAR’

) AND fooB.B >= ‘2011-08-12’

) AND fooA.A = 1

Posted by at 2 comments:

Thursday, August 11, 2011

Flex 3 Test Automation using QTP

This information has been sitting in my “to publish” list for a while now, so I am finally going to put it out there since a couple people have been asking about it. It describes my experiences with automating user interface interaction testing using QTP, and all the things I had to workaround.

1. Setting up and running QTP

1.1 Add “automation_charts.swf” to the Flex SDK directory

Example: C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs

1.2 Add the QTP libraries to your project compiler options

Add the following to the compiler options on the project:

-include-libraries "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\automation.swc" "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\automation_agent.swc" "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\automation_charts.swc" "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\qtp.swc"

The absolute path must be used, relative will not work.

This will force the libraries into your main application SWF, which should increase its size by about 1.2 MB

1.3 Run your Flex application on a web server.

You must run the application on a web server, it cannot run locally. If you attempt to run it locally you will get lots of browser scripting errors.

Example: http://myserver/foo/bar.html

1.4 You must be running Flash 9

The computer running the application and QTP must have Flash 9, otherwise system popups will be blocked by Flash Player security

1.5 You must run QTP and the browser window in the same monitor.

The computer running the application and QTP must be doing so in the same monitor window in the event that two monitors are being used, more on this later.

 

2. Required Code Changes

For various reasons when doing any type of user interface testing in Flex you have to make code changes.

2.1 Create new build and deployment

You will need to create a new variation of your build that compiles the automation libraries into the application binary (and all modules). This is because you don’t want to deploy your release binaries with the QTP automation classes compiled into them. This is because it significantly increases the size of your SWF.

2.2 Recursively set the automation hierarchy when running for automation

In order for the QTP to be able to playback button presses to objects deep within the component hierarchy, an automation value has to be set throughout the application at runtime, see “QTP Playback is slow” for information as to why and how.

2.3 Creation of automation delegates for every custom components that QTP does not recognized.

QTP will not recognized interactions with items that inherit directly from UIComponent directly, or are otherwise not a standard button, or text component, or control. See “Custom Automation Delegates” for more information.

 

3. QTP Specifics and Problems

3.1 Creating a Custom Flex Automation Delegate

References

Summary

Delegates exist so that you do not have to place automation related code in you standard components, after all, most people don’t want to run their applications with automation support. Delegates are provided for all the framework classes and will generally work out of the box for any framework class you extend. The need to write custom delegates arises if you create a custom component that directly extends UIComponent or if you have complex requirements for a component which already has a framework provided delegate.

A delegate has to be created for each custom class. Example:

package

{

import flash.display.DisplayObject;

import mx.core.IInvalidating;

import randomWalkClasses.RandomWalkEvent;

import mx.automation.IAutomationObject;

import mx.automation.AutomationIDPart;

import mx.automation.delegates.core.UIComponentAutomationImpl;

import mx.automation.IAutomationObjectHelper;

import mx.automation.Automation;

import mx.automation.events.AutomationRecordEvent;

import flash.events.Event;

[Mixin]

public class RandomWalkDelegate extends UIComponentAutomationImpl

{

       private var walker:RandomWalk

       public function RandomWalkDelegate(randomWalk:RandomWalk)

       {

               super(randomWalk);

               randomWalk.addEventListener(RandomWalkEvent.ITEM_CLICK, itemClickHandler);

               randomWalk.addEventListener(AutomationRecordEvent.RECORD, labelRecordHandler);

               walker = randomWalk;

       }

       public static function init(obj:DisplayObject):void

       {

              Automation.registerDelegateClass(RandomWalk, RandomWalkDelegate);

       }

       private function itemClickHandler(event:RandomWalkEvent):void

       {

               recordAutomatableEvent(event);

       }

       public function labelRecordHandler(event:AutomationRecordEvent):void

       {

               // if the event is not from the owning component reject it.    

               if(event.replayableEvent.target != uiComponent)

               //event.preventDefault(); can also be used.

              event.stopImmediatePropagation();

       }

       override public function get numAutomationChildren():int

       {

              var numChildren:int = 0;

              var renderers:Array = walker.getItemRenderers();

              for(var i:int = 0;i< renderers.length;i++)

              {

                numChildren += renderers[i].length;

              }

              return numChildren;

       }

       override public function getAutomationChildAt(index:int):IAutomationObject

       {

              var numChildren:int = 0;

              var renderers:Array = walker.getItemRenderers();

              for(var i:int = 0;i < renderers.length;i++)

              {

                  if(index >= numChildren)

                  {

                          if(i+1 < renderers.length && (numChildren + renderers[i].length) <= index)

                          {

                             numChildren += renderers[i].length;

                             continue;

                          }

                          var subIndex:int = index - numChildren;

                          var instances:Array = renderers[i];          

                          return (instances[subIndex] as IAutomationObject);

                   }

              }

       return null;

       }

       override public function createAutomationIDPart(child:IAutomationObject):Object

       {

               var help:IAutomationObjectHelper = Automation.automationObjectHelper;

               return help.helpCreateIDPart(this, child);  

       }  

       override public function resolveAutomationIDPart(part:Object):Array   

       {            

               var help:IAutomationObjectHelper = Automation.automationObjectHelper;

               return help.helpResolveIDPart(this, part as AutomationIDPart);  

       }

       override public function replayAutomatableEvent(event:Event):Boolean

       {

               var help:IAutomationObjectHelper = Automation.automationObjectHelper;

               if (event is RandomWalkEvent)

               {

                 var rEvent:RandomWalkEvent = event as RandomWalkEvent

                         help.replayClick(rEvent.itemRenderer);

                        (uiComponent as IInvalidating).validateNow();

                        return true;

               }

               else

               return super.replayAutomatableEvent(event);

       }

}

}

The TEAFlexCustom.xml for Mercury on all machines that intend to do automation has to be updated to include details about the custom class. Example:

<TypeInformation xsi:noNamespaceSchemaLocation="ClassesDefintions.xsd"

Priority="0" PackageName="TEA" Load="true" id="Flex" xmlns:xsi="http:/

/www.w3.org/2001/XMLSchema-instance">

       <ClassInfo Name="FlexRandomWalk" GenericTypeID="randomwalk" Extends="FlexObject" SupportsTabularData="false">

              <Description>FlexRandomWalk</Description>

              <Implementation Class="RandomWalk"/>

              <TypeInfo>

                      <Operation Name="Select" PropertyType="Method" ExposureLevel="CommonUsed">

                             <Implementation Class="randomWalkClasses::RandomWalkEvent" Type="itemClick"/>

                             <Argument Name="itemRenderer" IsMandatory="true" >

                                    <Type VariantType="String" Codec="automationObject"/>

                                    <Description>User clicked item</Description>

                             </Argument>

                      </Operation>

              </TypeInfo>

              <Properties>

                      <Property Name="automationClassName" ForDescription="true">

                             <Type VariantType="String"/>

                             <Description>To be written.</Description>

                      </Property>

                      <Property Name="automationName" ForDescription="true">

                             <Type VariantType="String"/>

                             <Description>The name used by the automation system to identify an object.</Description>

                      </Property>

                      <Property Name="className" ForDescription="true">

                             <Type VariantType="String"/>

                             <Description>To be written.</Description>

                      </Property>

                      <Property Name="id" ForDescription="true" ForVerification="true">

                             <Type VariantType="String"/>

                             <Description>Developer-assigned ID.</Description>

                      </Property>

                      <Property Name="automationIndex" ForDescription="true">

                             <Type VariantType="String"/>

                             <Description>The object's index relative to its parent.</Description>

                      </Property>

                      <Property Name="openChildrenCount" ForVerification="true" ForDefaultVerification="true">

                             <Type VariantType="Integer"/>

                             <Description>Number of children open currently</Description>

                      </Property>

              </Properties>

       </ClassInfo>

</TypeInformation>

3.2 File Selection fails to playback

When playing back the selection of a file in a System32 file dialog, the following error is thrown in QTP with the message “Object not visible”:

image041

Cause: Dual Monitors
Reference: http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1200371

Summary: This error is sometimes thrown when the browser, the secondary window (the file dialog), and QTP are not all in the same monitor.

Solution: Display all the windows in the same monitor.

3.3 QTP Playback is slow

Cause #1: Bug with automation in Flex 3 SDK

Summary: QTP playback is slower because of the implementation of automation libraries for Flex in the version 3 SDK. The issue does not exist in the version 2 SDK and has been resolved in the 4 SDK.

This is specifically because deep container hierarchies are slow to iterate through.

In the Flex 3 SDK automation scripts are significantly slower in deep component and container hierarchies. This is because the scripts have to iterate through the entire hierarchy in order to find the component that is being used. This issue was resolved in the Flex 4 SDK,

For example pressing the upload button to open the file dialog took 4 minutes, because the button is so deep within the hierarchy.

Potential Workaround: Set the automationName of the container and component

Workaround Summary

The automationName is a property available every component which designates the name that is used to identify a component in test automation. If no automationName is specified in Flex containers use their id attribute, and if no id attribute is present the container will have an automationName generated for it prefixed with “index”. Buttons however use the value of their label as their automationName when it is not specified.

It was theorized that setting the automationName may allow quicker access to the component, without having to iterate through the entire component container hierarchy, but this did not make a difference.

For example a button without an automationName would be accessed in QTP like the following:

Browser(#).FlexApplication(“Client”).FlexButton(“Upload and Print”).Click

Where “Upload and Print” is the generated automationName from the button label, but if automationName were set on the component in flex to “myButton” then the button would be accessed like the following:

Browser(#).FlexApplication(“Client”).FlexButton(“myButton”).Click

Actual Workaround: Set every container in the application hierarchy to showAutomationInHierarchy to true

References:

Workaround Summary

By default when QTP has to access a Flex component, it has to access that component by automationName. That component is located within an application by iterating starting with the application container and all of its children until the component within that automationName is found. The result is that a component that is deep within a container hierarchy is slow to find. It is for this reason and others, that it is recommend that Flex applications do not overuse container components. Unfortunately in the case of a large already built application, it is too late to go back and redesign the component container structure.

The solution is to specify every automationName within a hierarchy in order to limit the looping needed to find a component. This can be done by setting the showAutomtationInHierarchy value on a container to true, which will cause it to be detected in QTP even if that component is not directly involved in the interaction. If a Flex application uses modules to incrementally load itself over time, it is not possible on startup to recursively set the hierarchy value. Instead after a module is loaded, the module and the components within can recursively have this value set.

public function configureAutomation(obj:DisplayObject) : void

{

        if (obj is Container)

        {

            var c:Container = obj as Container;

                c.showInAutomationHierarchy = true;

                var children:Array = c.getChildren();

            for(var i:int=0; i < children.length; i++)

                {

                    var child:DisplayObject = c.getChildAt(i);

                    configureAutomation(child);

            }

        }

}

For example without this change the Upload button in the Client is accessed using QTP like the following, which requires 4 minutes to process:

Browser("#").FlexApplication("Client").FlexDividedBox("workflowUploadBox").FlexButton("uploadFirstBtn").Click

When all of the containers have the automation hierarchy set to show, that same button is accessed like the following in QTP and happens instantly:

Browser("#").FlexApplication("Client").FlexContainer("index:29").FlexCanvas("_DocumentViewModule_DocumentVi").FlexDividedBox("workflowUploadBox").FlexBox("index:0").FlexCanvas("newDocView").FlexBox("_NewDocument_VBox1").FlexCanvas("uploadView").FlexBox("_UploadView_HBox2").FlexButton("uploadFirstBtn").Click

Cause #2: QTP 9.1 is slow

Summary: Script playback in QTP 9.1 is very slow, and switching to 9.5 makes the playback significantly faster.

 

4. Maintainability Concerns

This is a concern with Flex and HTML based user interface drivers, in which the actor has to know the ID of the component in order to play back some action on it. In languages like JSF and Flex you don’t have to give an XML component an ID, which results in that component getting a dynamically generated ID.

image001

In Flex the problem is that when a component is not explicitly named, that name is calculated at compile time. The problem with the generated name is that it is based on a component’s position in its hierarchy. For example when a new Canvas is added, the names of the other two canvases change as shown by the highlighting in red.

It should also be noted that buttons are labeled by their text, which means of the text on the button changes the script will also have to change.

In order to ensure that a component’s name to QTP is always the same, the developer has to set the component’s automation name in the code for buttons and labels, and the component’s ID for boxes and canvases.

Posted by at No comments:
Subscribe to: Posts (Atom)

Contributors