Software Everything

Upgrading JBoss 4 to JBoss 5 with Java 5 to Java 6

2011-12-20T09:37:00.001-06:00

The information presented here comes from an effort to upgrade a Java enterprise application to the most current versions of all of its parts; primarily to get onto Java 6. Its starting system specifications were the following:

JBoss 4.2.2
Java 1.5.0_27
SEAM 2.1.0
RichFaces 3.3.1
JSF 1.2
EJB 3.0
Remote EJB’s

For reasons that will be explained in detail, it was not possible to upgrade each of the system parts to their various current versions. The end result was the following:

JBoss 5.1.0
Java 1.6.0_27
SEAM 2.2.2
RichFaces 3.3.3
JSF 1.2
EJB 3.0
Remote EJB’s

Due to the size and scope of the enterprise application itself in combination with the incompatibilities between system components, the entire effort ended up taking me a long time. The following is a log of this endeavor with all of the errors I encountered and how they were resolved. The intention of this writing is for both a historical purpose, and to help others avoid the pitfalls of performing such an upgrade on a sufficiently large enterprise application.

Why didn’t it just work out of the box with Java 6?

JBoss 4.2.2 Web Services do not work with Java 1.6. It results in a runtime error with SOAP. This means in order to run with Java 6 we have to upgrade JBoss.

java.lang.UnsupportedOperationException: setProperty must be overridden by all subclasses of SOAPMessage

What about JBoss 7.0?

The configuration is so different that it in no way resembles any previous version, and it does not support Remote EJB’s.

Remote EJB invocations isn't currently available in AS 7.0. That will be implemented by AS 7.1.
(http://community.jboss.org/message/616635)

So, how about JBoss 6.1.0?

JBoss 6 supports Java 6, SEAM, RichFaces, JSF, EJB 3.0, and remote EJB’s. The question is now what versions of these will work?

What about RichFaces 4?

Yes, it works but with JBoss 6.1.0 and JSF 2.0.

http://docs.jboss.org/richfaces/latest_4_0_X/Developer_Guide/en-US/html/chap-Developer_Guide-RichFaces_overview.html

RichFaces supports the following JSF implementations and frameworks:

MyFaces 2 and higher
Seam 3 and higher
Mojara 2 and higher

So looks like you have to upgrade to SEAM 3

Thus far we have JBoss 6.1.0, RichFaces 4.0.0, and JSF 2.0. SEAM 3 has apparently changed everything, and is not backward compatible with a SEAM 2 configuration. I was never able to even get “Hello World” working with SEAM 3, which is to be expected since it is “not done” (http://seamframework.org/157223.lace).

While according to compatibility it is supposed to work with JBoss 6 (http://seamframework.org/Seam3/Compatibility), I again haven’t been able to get even the most simple example working.

Back to SEAM 2

Since SEAM 3 does not work we have no choice but to go back to SEAM 2. RichFaces requires JSF 2.0 and in its specification says it only works with SEAM 3. While I question how one could know that RichFaces 4 only works with SEAM 3 when no one to my knowledge has a working example on JBoss 6, the goal is to upgrade to Java 6. There is also some unspecified hack to get SEAM 2 working with JSF 2, but any type of hack would probably make upgrading harder in the future. Okay, back to to RichFaces 3.

Back to RichFaces 3, how about 3.3.3?

Sorry no, not on JBoss 6: http://docs.jboss.org/richfaces/latest_3_3_X/en/devguide/html_single/#SupportedServers

JBoss 6 isn’t even listed as a supported server, and in my experience I haven’t been able to find anyone able to get RichFaces 3 working under JSF 1.2 or 2.0 on JBoss 6. I haven’t been able to get a “Hello World” working either.

Wait, where are we now?

We can’t use SEAM 3 because it doesn’t work (or at least the community can’t get it to work) on JBoss 6
We can’t use RichFaces 4 because it requires JSF 2 and SEAM 2 requires JSF 1.2
RichFaces 3 isn’t supported (and won’t run apparently) on JBoss 6
Finally, was can’t use JBoss 6 because SEAM 2 doesn’t support it (http://seamframework.org/Seam2/GettingStarted), and of course RichFaces 3 doesn’t work with it either

JBoss 5.1.0, Hello World

On paper RichFaces 3 and SEAM 2 work on JBoss 5, and JBoss 5 can run on Java 6. Take into account though the same is true for RichFaces 4, JSF 2, SEAM 3, and Remote EJB’s on JBoss 6. While an example exists for RichFaces 3 + SEAM 2 on JBoss, it of course doesn’t work out of the box on JBoss 5.

As a starting point I used Eclipse 3.7 (Indigo), JBoss Tools 3.3.3 (http://www.jboss.org/tools/download/installation/update_3_3), and Seam 2.2.2 (http://seamframework.org/Seam2/Downloads). JBoss Tools has a “Create Seam Web Project wizard,” so it should work right out of the box, right?

First, you have to rename jboss-seam-2.2.2.jar to jboss-seam.jar in the seam-2.2.2/lib installation. This is because if you don’t the SEAM wizard won’t allow you to select where you put SEAM because it looks for the JAR of this name.

Second, RichFaces component will not work out of the box. If you try to use something like a calendar or a TabPanel you will get this message:
Resources framework is not initialised, check web.xml for Filter configuration

Not too hard; add this to the web.xml:
<filter>
<display-name>Ajax4jsf Filter</display-name>
<filter-name>ajax4jsf</filter-name>
<filter-class>org.ajax4jsf.Filter</filter-class>
</filter>
<filter-mapping>
<filter-name>ajax4jsf</filter-name>
<url-pattern>*.seam</url-pattern>
<dispatcher>REQUEST</dispatcher>
<dispatcher>FORWARD</dispatcher>
<dispatcher>INCLUDE</dispatcher>
</filter-mapping>

Third, there are all sorts of problems with .xhtml files:

layout/template.xhtml - Delete anything referencing authentication
layout/menu.xhtml - Delete the content of the toolBarGroup

Fourth, you have to delete the xerces and xml-apis jars out of the lib directory. These are in JBoss now.

EARs containing Beans and WARs with shared libraries in the root of the EAR

A detail of the project I am converting is that it contains several WAR files, in which the library dependencies reside in the root level of the EAR, instead of the WEB-INF/lib directories of the WARs.

There are a couple of interesting side-effects of this type of structure on JBoss 5:

The app no longer runs in Eclipse, because the libraries don’t actually reside in the root directory of the deployment
Having these dependencies in the root of the EAR interferes with the admin-console deployment. There are several ways of dealing with this, but the easiest is just to remove it from deploy. (https://issues.jboss.org/browse/JBAS-7032)

In terms of getting this configuration to work in Eclipse, you only would have to automatically copy the JAR files to the root of the deployment. The deployment assembly settings right? No. When they work, they only copy a few files. The files they choose to copy whether you use a filterset or add each one manually are seemingly random.

How about a custom builder within Eclipse to copy the files? The JBoss server in eclipse deploys to a dynamic location within .metadata\.plugins\org.jboss.ide.eclipse.as.core within your workspace, so you would need to access it through an environment variable if you were trying to create custom builder. Is there an environment variable like ${jboss_config}? There would have to be but good look figuring it out.

In order to have JBoss in a predictable place, you can modify the server settings by double-clicking on the JBoss Server in the service view, selecting the Deployment tab, and making it deploy to a custom location (http://community.jboss.org/message/564310).

Since you don’t want your Eclipse version of your EAR messing with your manually deployed one, it is best to create a second server/YOUR_PROJECT deployment directory which is a copy of the default minus deploy/admin-console. You also have to make sure to modify the JBoss runtime configuration to NOT use the default configuration, and to use the one you just created.

For example under JBoss Runtime Configuration -> Open Launch Configuration::
--configuration=eclipse -b localhost -Djboss.server.base.url=file:/C:/Java/jboss-5.1.0.GA/server/

The story of our app...

Jars over, application.xml changed, JBoss properties-service.xml changed to use environment.properties, build changed to use Java 6 and the JBoss 5 libraries, Eclipse project modified to compile using all the new libraries, and all of the other modifications made according to “Hello World”. It should deploy just fine right?

Error #1: Error installing to PostClassLoader

[org.jboss.kernel.plugins.dependency.AbstractKernelController] (main) Error installing to PostClassLoader: name=vfsfile:/C:/Java/jboss-5.1.0.GA/server/myapp/deploy/MyApp.ear/ state=ClassLoader mode=Manual requiredState=PostClassLoader
org.jboss.deployers.spi.DeploymentException: Error during deploy: vfszip:/C:/Java/jboss-5.1.0.GA/server/myapp/deploy/MyApp.ear/account-application.jar/
    at org.jboss.deployers.spi.DeploymentException.rethrowAsDeploymentException(DeploymentException.java:49)
…
Caused by: java.lang.NullPointerException
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:247)
    at org.jboss.metadata.process.processor.ejb.jboss.SetDefaultLocalBusinessInterfaceProcessor.process(SetDefaultLocalBusinessInterfaceProcessor.java:114)
    at org.jboss.metadata.process.chain.ejb.jboss.JBossMetaDataProcessorChain.process(JBossMetaDataProcessorChain.java:115)
    at org.jboss.ejb3.deployers.Ejb3MetadataProcessingDeployer.deploy(Ejb3MetadataProcessingDeployer.java:115)
    at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)

Possibilities:

A problem with the version declared in the ejb-jar.xml: https://issues.jboss.org/browse/EJBTHREE-1976. Nope, version is 3.0 in the HelloWorld as well
The jboss.xml could be referencing a bean that is a pojo (https://issues.jboss.org/browse/JBMETA-195). While commenting out the beans got the deployment farther, it didn’t get rid of this error. This indicates this is a problem with something else.
EJB’s with zero parameters: https://issues.jboss.org/browse/JBMETA-207. This seems pretty likely since the version of JBoss listed is 5.1.0

This stack trace is pretty useless, and the only matches for any part of it on the internet are to those two JIRA issues. Resorting to just a “how to go from JBoss 4 to 5” search:

Persitence.xml and ejb-jar.xml must validate, remove xalan.jar, xercesImpl.jar, serializer.jar, xml-apis.jar from the application. (http://community.jboss.org/wiki/MigrationFromJBoss4)
There are numerous guides relating to this; too many to list

Nothing works, now what?

An issue with all these guides though is that they are for applications most likely smaller than the one we are upgrading. Sticking will all the advice still doesn’t have an effect on this startup error. The next resort is to break our application, which currently sits as one big Java project that can only be run by ant build and deploy, into its component pieces and get them working one at a time. Starting with the EJB that contains all the Java code that is causing this error.

Our project’s directory structure is the following:

Application

src (EJB)
ear
WebEar

Application WAR #1
Application WAR #2
Application WAR #3
Application WAR #4

I have experimented with running this configuration out of Eclipse, and while it is possible the configuration is extremely complicated. I however also like to avoid developer setup documents which are 10 pages long, so sometimes the path of least resistance is the best way to go.

The advantage of being able to run out of Eclipse is that you don’t have to Ant build, deploy it, and startup JBoss every time you want to see a change. A different project structure also lets things be broken apart and tested more easily, because they don’t rely on you having to modify an already complicated Ant build.

The idea is that like “Hello World,” a project that we can get running easily in Eclipse should be in this project directory structure:

Application EAR (Enterprise Application Project)

Application Bean (EJB Project with no client)
Application WAR #1 (Dynamic Web Project)
Application WAR #2 (Dynamic Web Project)
Application WAR #3 (Dynamic Web Project)
Application WAR #4 (Dynamic Web Project)

The reliance on the Eclipse project wizards results in projects that run in Eclipse out-of-the-box.

Checklist

EAR deploys with Bean
EAR deploys with Bean and application.xml contains all of the needed libraries
EAR deploys with Bean, libraries, and data sources
Error: EAR deploys with Bean, libraries, data sources, and messaging config

Error #2: Queue[/queue/myapp/bulkMessageQueue

ERROR [ExceptionUtil] Queue[/queue/myapp/bulkMessageQueue, name=myapp/bulkMessageQueue] startService
javax.naming.NameNotFoundException: myapp not bound

The issue is that JMS configuration has changed between 4 and 5: http://javabeanz.wordpress.com/2009/06/05/configuring-jms-in-jboss-5/

For myapp-jms-service.xml:

JBoss 4:

<mbean code="org.jboss.mq.server.jmx.Queue"
    name="jboss.mq.destination:service=Queue,name=myapp/bulkMessageQueue">
        <depends optional-attribute-name="DestinationManager">
            jboss.mq:service=DestinationManager
        </depends>
</mbean>

JBoss 5:

<mbean code="org.jboss.jms.server.destination.QueueService"
     name="jboss.messaging.destination:service=Queue,name=myapp/bulkMessageQueue"
     xmbean-dd="xmdesc/Queue-xmbean.xml">
     <depends optional-attribute-name="ServerPeer">
        jboss.messaging:service=ServerPeer
     </depends>
     <depends>jboss.mq:service=DestinationManager</depends>
     <attribute name="JNDIName">queue/myapp/bulkMessageQueue</attribute>
</mbean>

Back to the checklist:

EAR deploys with Bean, libraries, data sources, messaging config, and mail config
… and the Seam interceptors in the ejb-jar.xml
ERROR: … add all the source code for the bean

Back to ERROR #1: Error installing to PostClassLoader

org.jboss.kernel.plugins.dependency.AbstractKernelController] (main) Error installing to PostClassLoader: name=vfsfile:/C:/Java/jboss-5.1.0.GA/server/myapp/deploy/MyApp.ear/ state=ClassLoader mode=Manual requiredState=PostClassLoader

We now know that this error has something to to with one or more of the following:

Java source - Can’t really remove this
jboss.xml - Removing this gets rid of the error
ehcache.xml
persistence-container.xml
The WSDL files

Ding, ding, ding! Something is wrong with the jboss.xml

jboss.xml looks like this:

<jboss>
    <enterprise-beans>
        
        <session>
            <ejb-name>BasketActionImpl</ejb-name>
            <jndi-name></jndi-name>
            <clustered>false</clustered>
        </session>
        <session>
            <ejb-name>ContractPricingActionImpl</ejb-name>
            <clustered>false</clustered>
        </session>
...

The presence of one or more of those beans is what is causing the error. A cursory look has also revealed that several of the beans here don’t exist. I eventually found out that the reason for the failures was related to ejb-name classes that don’t exist. After removing classes (one at a time) jboss was able to launch with this configuration. Note though that removing the clustering settings causes error #12.

ERROR #3: Can't find a persistence unit

org.jboss.deployers.spi.DeploymentException: Error deploying MyEJB.jar: Exception while processing container metadata for EJB: DBServiceImpl in unit: MyEJB.jar
…
Caused by: java.lang.IllegalArgumentException: Can't find a persistence unit named 'MyPu' in AbstractVFSDeploymentContext@31521104{vfsfile:/C:/Java/jboss-5.1.0.GA/server/eclipse/deploy/MyApp.ear/MyEJBjar/}

There is obviously a problem with the persistence-container.xml, as it is the one that defines the datasource which cannot be found. According to the thread at http://community.jboss.org/thread/146523, there are a couple different ways to deal with persistence.xml. Note that is is always referred to as “persistence.xml”. If you change the name of the this XML file to persistence.xml, it works. This is an indication that JBoss 5 wants your persistence unit to be defined in persistence.xml by default.

Back to the checklist:

...and WAR #1 with a relatively empty configuration
ERROR: ...and WAR #1 with RichFaces and SEAM in the configuration from Hello World

Error #4: NoClassDefFoundError: org/jboss/resteasy/specimpl/UriInfoImpl

java.lang.ClassNotFoundException: javax.ws.rs.ext.Provider
java.lang.NoClassDefFoundError: org/jboss/resteasy/specimpl/UriInfoImpl

This is because of the inclusion of jboss-seam-resteasy-2.2.2.jar, which does not include the REST Easy dependencies. This can just be removed from the application.xml.

Error #5: Server cycles between deploying and undeploying

2011-10-17 12:14:03,137 INFO [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (HDScanner) deploy, ctxPath=/account
2011-10-17 12:14:03,199 INFO [org.apache.catalina.startup.ContextConfig] (HDScanner) WARNING: Security role name NONE used in an <auth-constraint> without being defined in a <security-role>
2011-10-17 12:14:03,215 INFO [javax.enterprise.resource.webcontainer.jsf.config] (HDScanner) Initializing Mojarra (1.2_12-b01-FCS) for context '/account'
2011-10-17 12:14:05,324 INFO [javax.servlet.ServletContextListener] (HDScanner) Welcome to Seam 2.2.2.Final
2011-10-17 12:14:09,434 WARN [org.jboss.seam.security.permission.PersistentPermissionResolver] (HDScanner) no permission store available - please install a PermissionStore with the name 'org.jboss.seam.security.jpaPermissionStore' if persistent permissions are required.
2011-10-17 12:14:09,621 INFO [org.quartz.simpl.SimpleThreadPool] (HDScanner) Job execution threads will use class loader of thread: HDScanner
2011-10-17 12:14:09,637 INFO [org.quartz.core.QuartzScheduler] (HDScanner) Quartz Scheduler v.1.5.2 created.
2011-10-17 12:14:09,637 INFO [org.quartz.simpl.RAMJobStore] (HDScanner) RAMJobStore initialized.
2011-10-17 12:14:09,637 INFO [org.quartz.impl.StdSchedulerFactory] (HDScanner) Quartz scheduler 'DefaultQuartzScheduler' initialized from default resource file in Quartz package: 'quartz.properties'
2011-10-17 12:14:09,637 INFO [org.quartz.impl.StdSchedulerFactory] (HDScanner) Quartz scheduler version: 1.5.2
2011-10-17 12:14:09,637 INFO [org.quartz.core.QuartzScheduler] (HDScanner) Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
2011-10-17 12:14:09,902 WARN [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (HDScanner) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jboss/cache/pojo/jmx/PojoCacheJmxWrapperMBean
2011-10-17 12:14:20,871 INFO [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (HDScanner) undeploy, ctxPath=/account

I encountered this issue earlier and just blamed it on Eclipse, but closer attention to the logs reveals there is a missing class. This JAR can be downloaded here: http://www.java2s.com/Code/Jar/JKL/Downloadpojocachejar.htm

Followed by:
2011-10-17 13:15:05,809 WARN [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jboss/cache/jmx/LifeCycle

Apparently life has moved on past JBoss Cache, http://community.jboss.org/wiki/JBossCache, but you can get jbosscache here: http://www.java2s.com/Code/Jar/JKL/Downloadjbosscachejar.htm

Followed by:
2011-10-17 13:23:37,277 WARN [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jboss/cache/CacheManager

The JAR can be downloaded here: http://www.java2s.com/Code/Jar/JKL/Downloadjbosscachecorejar.htm

Followed By:
2011-10-17 14:14:10,356 WARN [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. NoClassDefFoundError: org/jgroups/blocks/MethodCall

jgroups.jar, which can be obtained in the JBoss Cache release.

Followed by:
2011-10-17 13:34:08,809 WARN [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) Failed to setup clustering, clustering disabled. ClusteringNotSupportedException: Could not access CacheManager for JBossWeb clustering

At this point I am baffled, as there are not any other obvious signs of why JBoss decided it doesn’t like this context and undeploys it. The message “Could not access CacheManager for JBossWeb clustering” on Google only points to the source code from where it came, so I am either way off or this is not related.

Upon further research, these messages have nothing to do with the undeploy. There is a majorly annoying bug in all of JBoss 5: http://stackoverflow.com/questions/1451614/jboss-deploy-undeploy-loop-caused-by-hdscanner-in-a-exploded-ear

You pretty much can’t have anything but .xml files in your WAR WEB-INF directory. Removing a .jpg from here resolved this issue.

Error #6: SEAM Actions don’t resolve

2011-10-17 15:08:59,774 WARN [org.jboss.ejb3.interceptors.aop.InterceptorsFactory] (http-localhost%2F127.0.0.1-8080-1) EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container
2011-10-17 15:08:59,805 WARN [org.jboss.ejb3.interceptors.aop.InterceptorsFactory] (http-localhost%2F127.0.0.1-8080-1) EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container
2011-10-17 15:09:13,555 WARN [org.jboss.seam.security.jaas.SeamLoginModule] (http-localhost%2F127.0.0.1-8080-2) Error invoking login method
javax.el.PropertyNotFoundException: Target Unreachable, identifier 'loginAction' resolved to null

Apparently there is a problem with the SEAM security configuration, which is what controls login. This is to be expected since security changed in SEAM 2.2.2: http://docs.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/5/html/Seam_Reference_Guide/Migration20.Changes.html

There is also of course different XSD’s now available, which require that you change all the -2.1.xsd’s to 2.2.xsd’s.

The following post indicates that the problem may be that the class inside the JAR within the EAR is not being picked up by SEAM, because there is no seam.properties files in the JAR: http://stackoverflow.com/questions/3978976/can-the-seam-authenticator-reside-in-the-ear-file-lib-folder

This was the problem. If you have beans in a JAR that JAR must define an empty seam.properties file in order for it to be scanned for beans.

Error #7: javax.naming.NameAlreadyBoundException: classPathEntries

Apparently there is an issue with the hibernate-core jar existing in both the ears in the deployment, this was resolved by deleting deleting the second ear. Eventually the second EAR was made to not include this JAR.
Also had the same problem with javassist

Error #8: Unit Tests no longer run out of Eclipse

The problem is running unit tests with a classpath that is too long in Eclipse.

You have to edit the classpath of the target to not include external dependencies

Error #9: Some pages are 404 not found

This was a really annoying problem that plagued me for a long time. Buttons that used to correctly direct to htm pages were no longer working. Upon closer inspection the link that works is going to pageC.htm while the one that doesn’t is going to pageB.xhtml. Something is not rewriting xhtml to htm in the page rendering.

The problem the entire time has been these pages were named “xtml” instead of “xhtml” and I didn’t see it. Apparently the previous version of SEAM, JBoss, Java, or something else handled xtml even though it is not a part of the configuration.

Error #10: java.lang.InstantiationException: org.apache.log4j.Logger

javax.ejb.EJBException: java.lang.RuntimeException: org.jboss.serial.exception.SerializationException: Could not create instance of org.apache.log4j.Logger - org.apache.log4j.Logger
...
caused by: java.lang.InstantiationException: org.apache.log4j.Logger

Google only provided two solutions: make the logger static or declare it as @Transient

Doing both of these didn’t get rid of the error.

There was a warning about a @Destory method in a class that was using a log4j logger, so I changed it to be seam injected. This too didn’t work, even when marked with @Transient.

As it turns out the way to fix this is simple, only use static transient Loggers. In my application this means I had to do this 890 times. A result of this problem is also a memory leak, because the beans can’t be destroyed.

Error #11: EJB Timer throws a null pointer exception

2011-10-28 09:19:54,584 ERROR [org.jboss.ejb.txtimer.TimerImpl] (EJB-Timer-1319811479419[target=jboss.j2ee:ear=Irio.ear,jar=jboss-seam-2.2.2.jar,name=TimerServiceDispatcher,service=EJB3]) Error invoking ejbTimeout
javax.ejb.EJBException: java.lang.RuntimeException: java.lang.NullPointerException

It doesn’t prevent startup and corrects itself, but slows the start up process down. There doesn’t appear to be anyway to get rid of getting this exception at least once on start up.

However, it should be noted that if you start getting these errors several times during startup it is an indication that JBoss will fail to startup eventually. The only way to prevent this is to delete the tmp, work, and data directories every time you restart a JBoss instance.

Error #12: Caused by: java.lang.NoClassDefFoundError: org/jgroups/Channel

The solution is to add the jgroups jars to lib/endorsed for whatever server you are running.

Reference: http://community.jboss.org/message/440606

But then you get a stack:

2011-11-03 09:57:33,359 ERROR [org.jboss.ejb3.cache.simple.SimpleStatefulCache.SMSOptionsActionImpl] (SFSB Passivation Thread - jboss.j2ee:ear=Irio.ear,jar=IrioAccountApplication.jar,name=SMSOptionsActionImpl,service=EJB3) problem passivation thread
javax.ejb.EJBException: Could not passivate; failed to save state

Reference: http://seamframework.org/Documentation/HandlingPrePassivateCleanupInSFSBs

Reading this article lead me to take a look at clustering again, which has to do with Error #1. As is turned out setting the troublesome class to clustered = false in the jboss.xml seems to have resolved this issue.

I again had to come back to this, and found that further down in the stacktrace there is a message about a missing class.

Long story short, I had to add the following jars to lib/endorsed:

mail.jar
log4j.jar
commons-logging.jar
jbosscache.jar
jgroups-all.jar
pojocache.jar
jbosscache-core.jar

The error following the passivation error, was java.lang.NoClassDefFoundError: org/jboss/cache/pojo/jmx/PojoCacheJmxWrapper

When I looked in the logs prior to all of this and during startup, I found that there was a warning message stating the clustering was disabled because orjava.lang.NoClassDefFoundError: org/jboss/cache/pojo/jmx/PojoCacheJmxWrapper. So I figured if I got rid of this startup error that it would get rid of the passivation error. The problem though is that they only way to get this error is to do something involving the bean, and then wait several hours.

Error #13: Database connection hangs forever (using MS SQL)

The reason for this has to do with a problem between Java 1.6.0_29 and the MS SQL JDBC driver. The only solution is to use Java 1.6.0_27, which is why that is the version of Java listed in the new spec even though it is not the most recent version of Java 6.

Reference: http://stackoverflow.com/questions/7841411/driver-getconnection-hangs-using-sqlserver-driver-and-java-1-6-0-29

Error #14: JBOSS_HOME and JAVA_HOME environment variables do to propagate into scheduled tasks

http://community.installshield.com/showthread.php?t=147931 indicates that the only way Scheduled Tasks can pick up changes to environment variables is to reboot the machine. However, in my case I was not able to reboot the machine so I had to find a workaround.

To do it without having to reboot:

Login as the non-running user
Disable all Scheduled Tasks
Change the environment variable
Start the Task Manager and select to show all processes from all users
In the Task Manager, kill all tasks being run by the Scheduled Task running user (taskeng.exe)
Log Off
Log on as the non-running user
Enable all Scheduled Tasks

ERROR #15: java.lang.OutOfMemoryError: unable to create new native thread

This issue has to do with the following things:

The amount of memory allocated with Xmx. This is in your conf if using the service wrapper.
The amount of memory allocated with Xms. This is in your conf if using the service wrapper.
The maximum number of AJP threads allowed in jbossweb.sar/server.xml

The key is finding the balance between these values, which is dependent on Java version, what your application is doing, and how much memory is available on your machine.

Reference: http://blog.egilh.com/2006/06/2811aspx.html

Error #16: Failed to parse source: {http://java.sun.com/xml/ns/javaee}virtual-host

org.jboss.xb.binding.JBossXBException: Failed to parse source: {http://java.sun.com/xml/ns/javaee}virtual-host cannot appear in this position. Expected content of {http://java.sun.com/xml/ns/javaee}web is unordered_sequence: {http://java.sun.com/xml/ns/javaee}context-root? {http://java.sun.com/xml/ns/javaee}web-uri?

The following is no longer valid, as virtual-host isn’t allowed here.
<module>
      <web>
         <web-uri>foo.war</web-uri>
         <context-root>/foo</context-root>
         <virtual-host>myhost</virtual-host>
      </web>
   </module>

Remove it because it should be defined in jboss-web.xml anyways.

Error #17: javax.ejb.EJBNoSuchObjectException: Could not find Stateful bean: 41z4l26-wb41k6-ev4j007b-1-ev4jirql-bw

I have been seeing this error in the logs quite often and in the previous version of JBoss, being 4.2.2. It didn’t appear to be critical so I jut let it be like the hundreds of other errors. I however stumbled across and article explaining the cause for this: http://community.jboss.org/wiki/JbossTimeoutSettingForSeam

To summarize, you get this problem with stateful session bean timeouts do not match the session timeout. This is great considering that the default session timeout (web.xml) is 30 minutes while the default stateful session bean timeout is 5 minutes (ejb3-intercepters-aop.xml).

Warning #1: EJBTHREE-1246: Do not use InterceptorsFactory with a ManagedObjectAdvisor, InterceptorRegistry should be used via the bean container

This is something that can be ignored entirely, and the developers even recommend that you do. Every time you access a bean you will get this warning, so it floods the log.

Reference: http://stackoverflow.com/questions/491007/jboss-what-does-the-warning-ejbthree-1246-from-the-interceptorregistry-mean

Using DBUnit through Ant with a large MS SQL Database

2011-08-23T08:26:00.001-06:00

The following describes my experiences with using DBUnit through Ant with a large MS SQL database, and all of the adventures I had. The intention is to use DBUnit though Ant to export a database as an XML file, to clear the database, and to re-populate it using that XML file.

Why?

Automated acceptance testing. You start with an XML backup of what you want your base system to contain, refresh your database with that backup, run your suite of automated acceptance tests, and then restore the XML backup again. That way your tests can do things specific to the application like create users, invoices, etc, and otherwise mess around with every conceivable operation. When you are done you can restore everything to the way it was before.

Running it though Ant lets you easily integrate this refresh into other automation tools such as Jenkins (Hudson), TeamCity, and so on.

Did you say large?

Yes I did, but in this context I am using it to refer to a database with a large number of tables, not a large number of records. A large number of tables is a catch-all for several circumstances which you cause you trouble, such as with tables which have keywords for names, foreign key dependencies between tables, tables with timestamp columns, and some more items I will get address.

But I want a lot of records!

There are a couple of problems with using a lot of records with DBUnit, the first of which is that if you are exporting and importing you will have to deal with very long save and load times. As a general rule you probably are aiming for an export XML file size of under 3 MB, which is around 2,000 records depending on what it is your tables. The second problem has to do with memory, but I have a workaround for that as well.

DBUnit with Ant

So you want to use Ant? You are then going to have to write your own task by extending the DBUnit Ant Task. This is because there are options that need to be set [at least to my knowledge] that cannot be set in the DBUnit Ant Task that is provided to you.

To start out you will need to extend three things:

The DbUnitTask, which is the <dbunit> tag. This allows you to define your own Export and Operation tags.
The Export, which is the <export> tag. This is for changing how the export works.
The Operation, which is the <operation> tag. This is for changing how the insert works.

MSDBUnitTask.java

public class MSDBUnitTask extends DbUnitTask {

    @SuppressWarnings("unchecked")
    public void addMSExport(MSExport export) {
        this.getSteps().add(export);
    }
    @SuppressWarnings("unchecked")
    public void addMSOperation(MSOperation operation) {
        this.getSteps().add(operation);
    }
}

MSExport.java

public class MSExport extends Export   {

    private String tableNames;
    @Override
    public void execute(IDatabaseConnection arg0) throws DatabaseUnitException {
        // code goes here
    }

    public String getTableNames() {
        return tableNames;
    }

    public void setTableNames(String tableNames) {
        this.tableNames = tableNames;
    }

}

MSOperation.java

public class MSOperation extends Operation   {

        @Override
    public void execute(IDatabaseConnection arg0) throws DatabaseUnitException {
        // code goes here
    }

}

Including your custom task in Ant

This assumes that the Ant classes from above are compiled into a JAR and present in the ${lib.dir} along with the dbunit.jar

Adventures in Exporting

The Java code here goes in the execute method of MSExport.

This is where the majority of problems start. As it turns out you can export with ease, but can end up with XML that had invalid characters and some other interesting things that cannot be imported.

Let’s start with the result we want (and need): the XML to export the database:

</target>

The msdbunit and msexport tags were explained earlier, and the base Java code for creating them was given.

Issue #1: Export Table Order

You must specify the order of tables to do the export in insertion order. This is done by passing in a comma separated list of tables name from ${insert.table.names} into the <msexport> tag as the tableNames attribute.

The reason for this order is that you must take into account FK dependencies when both inserting and deleting. For example if Foo.A is a FK to Bar.A, then you can’t start out by inserting records into Foo; you have to insert into Bar first. From the deletion perspective it is the opposite: You can’t delete from Bar since Foo can reference it and cause the delete to fail.

The MSExport task already has tableNames defined, so in order to get the result as a list of tables all that needs to be done is this:

String[] tableNamesArray = tableNames.split(",");

ITableFilter filter = new SequenceTableFilter(tableNamesArray);

IDataSet dataset = new FilteredDataSet(filter, arg0.createDataSet());

The second part of this is do use this array to define the table sequence. You must do this manually because even though it can be done automatically, with 100+ tables it will take minutes as opposed to a fraction of a second.

Issue #2: Columns that are of type TIMESTAMP or default DATETIME

There is an issue if you export a column of the TIMESTAMP type or of the defaulted DATETIME, because when you try and import it you will get an error regarding not being able to insert a value into the column.

The exact error you get is the following:

Cannot insert an explicit value into a timestamp column

The solution is not to export these columns, which requires the use of a column filter. This was the primary reason for abandoning the default Ant task, as I couldn’t figure out a good way to do a large column filtering.

List<ITable> updatedTables = new ArrayList<ITable>();

for (String tableName : tableNamesArray) {
                ITable table = dataset.getTable(tableName);
                List<Column> excludedColumns = new ArrayList<Column>();
                Column[] columns = table.getTableMetaData().getColumns();
                //For each column...
                for (Column column : columns) {
                    //TIMESTAMP columns have to be ignored in MSSQL because they cannot have an explicit insert
                    //Unfortunately DATETIME in MSSQL also shows here as a TIMESTAMP so the only way to
                    //determine defaulted DATETIME or TIMESTAMP is by looking for defaults and the SQL Type name
                    if ( (column.getDefaultValue() != null && column.getDataType() == DataType.TIMESTAMP) ||
                            column.getSqlTypeName().equals("timestamp")) {
                        excludedColumns.add(column);
                    }
                }
                //convert the list of excluded columns to an array
                Column[] excluded = excludedColumns.toArray(new Column[excludedColumns.size()]);
                //create a new ITable that excludes the filtered columns
                ITable filteredTable = DefaultColumnFilter.excludedColumnsTable(table,excluded);
                //keep track of the modified table
                updatedTables.add(filteredTable);
            }
            //Take all of the modified table and create a new dataset
            ITable[] updateTableArray = updatedTables.toArray(new ITable[updatedTables.size()]);
            CompositeDataSet compositeDataSet = new CompositeDataSet(updateTableArray);

This code iterates over all of the tables, looks for datetime and timestamp columns, excludes them from that table, and adds them to a new list of tables with the appropriate columns excluded. It then constructs a new composite dataset containing all of the new tables and columns.

Issue #3: Exporting your customizations

When you extend the Export task, you have to then define how to export.

//Get the format and file destination
String format = this.getFormat();
File dest = this.getDest();
//Output based on the format
if (format.equalsIgnoreCase(FORMAT_CSV)) {
    CsvDataSetWriter.write(compositeDataSet, dest);
else if (format.equalsIgnoreCase(FORMAT_FLAT)) {
    FlatXmlDataSet.write(compositeDataSet, new FileOutputStream(dest));
} else if (format.equalsIgnoreCase(FORMAT_XML)) {
    XmlDataSet.write(compositeDataSet, new FileOutputStream(dest));
} else if (format.equalsIgnoreCase(FORMAT_DTD)) {
    FlatDtdDataSet.write(dataset, new FileOutputStream(dest));
} else {
    throw new DatabaseUnitException(format+" is not a recognized format.");
}

Issue #4: Exporting Tables that have keywords for names

For example User is a keyword, but you can have a table of this name in MS SQL and access it using [User]. In order to deal with this you want to put all table names within [].

This can be configured in the Ant XML using the escapePattern property of dbconfig:

</target>

Issue #5: Data Type Factory

In order to be able to insert records into a MS SQL database using DBUnit, you have to specify the data factory to use as a part of the export. If you don’t any time you try and import a result containing a primary key you will get the following error:

Cannot insert explicit value for identity column in table

This issue can also be resolved in the default Ant using the dataTypeFactory property in the dbconfig:

</target>

Issue #6: Which table failed to export?

This issue is that when you are exporting your database and something goes wrong, it is not specified which table was the problem. You just get an error message such as “Cannot insert explicit value for identity column in table” and are left to figure out which table may have caused it.

Since we have our own Ant task that iterates over tables, this is pretty easy to do: put a System.out.println with the table name at the start of the loop. That was if something breaks, you know which table caused it.

Deleting from the Database

If you exported correctly this isn’t a problem, and doesn’t require any custom work beyond the escapePattern and data type factory. It should be noted that before inserting into your database you want to clear it using your export file first. The CLEAN_INSERT operation is supposed to do this on insert for you, but in my experience I have never been able to get it work on a large database. It ends up throwing constraint violations, either because of the database, a bug, or my misuse of the operation.

</target>

Populating the Database

The Java code here goes in the execute method of MSOperation.

The same as with deleting from the database, if you correctly exported the dataset you should not have any problems. There is one feature though that is nice to have, which is the ability in the case of an error to know which table cause the error.

        //Only perform an operation for every table if doing a clean insert for MS SQL
        if (this.getType().equalsIgnoreCase("MSSQL_CLEAN_INSERT")) {
            try {
                FileReader reader = new FileReader(this.getSrc());
                String format = this.getFormat();
                IDataSet dataset = null;
                if (format.equalsIgnoreCase(FORMAT_XML)) {
                    dataset = new XmlDataSet(reader);
                } else {
                    throw new DatabaseUnitException("xml is the only supported format.");
                }
                String[] tableNames = dataset.getTableNames();
                //For each table, create a new dataset and execute the operation because:
                for (int i = 0; i < tableNames.length; i++) {
                    String tableName = tableNames[i];
                    System.out.println("Populating table "+tableName);
                    ITable table = dataset.getTable(tableName);
                    ITable[] iTables = new ITable[1];
                    iTables[0] = table;
                    CompositeDataSet composite = new CompositeDataSet(iTables);
                    this.getDbOperation().execute(connection, composite);
                }
            } catch (Exception e) {
                e.printStackTrace();
                throw new DatabaseUnitException(e);
            }
        } else {
            super.execute(connection);
        }

}

Beyond that all you need is the following:

MSSQL_CLEAN_INSERT
The MS SQL Data Type Factory
The escapePattern for tables with keywords for names

</target>

Where did I get my information?

The internet combined with trial and error. Here are the sites I used for information:

Managing test data for integration tests using Spring and DBUnit (http://www.theserverside.com/news/1363748/Manage-test-data-for-integration-tests-using-Spring-and-DBunit)
Using DBUnit (http://www.fm-berger.de/ora_webtest/ch03.php)
DBUnit Tip: Turn off FK constraints (http://raibledesigns.com/rd/entry/dbunit_tip_turn_off_foreign)
DBUnit FAQ (http://ds.cc.yamaguchi-u.ac.jp/~joji/doc/dbunit/faq.html)
Column Filtering (http://old.nabble.com/comparing-actual-to-expected-tables-td17344245.html)
DBUnit Testing Framework (http://dbunit.sourceforge.net/xref-test/org/dbunit/ext/mssql/InsertIdentityOperationIT.html)
Using SelectMethod=cursor on a connection (http://support.microsoft.com/kb/917054)

SQL query performance using a select on a large record set with Hibernate

2011-08-16T13:34:00.001-06:00

This title is partly a question because I am unsure that my solution is the best. I have come to the following solution based on trial and error, and settling on what seemed to have worked.

The Database

The database is MS SQL 2008 Enterprise, we are using the correct JDBC driver, and are using Hibernate 3.4.0.GA. The database is also located on the same physical machine as the application server.

The Table

There is only a single table that is the target of the problem, which has the following structure:

Foo

ID (UUID Primary Key)
A (int, indexed)
B (datetime, indexed)
C (varchar, indexed)
D (int, not indexed)

In this table there are slightly over 2.5 million records, and I am trying to select less than 10 results.

The Query

This is the query that was slow, taking over 5 minutes to complete (in the SQL Variant). Take into account though that the different columns were used under different circumstances, making the query dynamic. The more columns used, the slower the query. In the case of using every column it looked like this:

SQL:

SELECT ID, A, B, C, D

FROM FOO

WHERE A = 1 AND B >= ‘2011-08-12’ AND C = ‘BAR’ AND D = 3

Hibernate:

Query query = em.createQuery("from Foo where a=:a and b >= :b and c = :c and d = :d”);

query.setParameter(“a”, 1);

query.setParameter(“b”, date);

query.setParameter(“c”, “BAR”);

query.setParameter(“d”, 3);

List<Foo> list = query.getResultList();

The Obvious

Column D isn’t indexed. Column D isn’t also used all of the time, and removing it from the query only cuts the 5 minute time by 30 seconds. There is also the preference that I do not change the existing database unless absolutely the only option.

There are also probably better ways to deal with the datetime column type, for example see http://diegworld.blogspot.com/2010/09/sql-server-performance-querying-by-date.html. However, if I remove this column from the query it doesn’t change the amount of time it takes.

The SQL Solution

Eventually I just started playing around with the SQL variant of the query to see if I could improve the performance. I started with running select statements with each individual column and found that they all worked in a second, for example:

SELECT ID, A, B, C, D, FROM FOO WHERE A = 1

SELECT ID, A, B, C, D, FROM FOO WHERE B >= ‘2011-08-12’

SELECT ID, A, B, C, D, FROM FOO WHERE C = ‘BAR’

SELECT ID, A, B, C, D, FROM FOO WHERE D = 3

At this point I realized that the problem has to do with all of the restrictions I added in the where clause, but changing the order of the “ands” didn’t improve performance either. With this information I then tried building the query so that it would incorporate sub queries. The idea being that if I have select with A I have set A, in which I can select with B from that and have Set AB, select C from that and have Set ABC, and select D from that and have Set ABCD. The resulting Set of ABCD would contain only records which had the A, B, C, and D criteria.

The resulting SQL that I came up with was the following:

SELECT ID, A, B, C, D FROM FOO fooA WHERE fooA.ID IN (

SELECT ID FROM FOO fooB WHERE fooB.ID IN (

    SELECT ID FROM FOO fooC WHERE fooC.ID IN (

          SELECT ID FROM FOO fooD WHERE D = 3

    ) AND fooC.C = ‘BAR’

) AND fooB.B >= ‘2011-08-12’

) AND fooA.A = 1

This query took less than a second to run, as opposed to the previous variant that was taking more than 5 minutes.

My question is though, is this the best thing to do from both a SQL and MSSQL perspective?

The Hibernate Implementation

Hibernate has a lot of flexibility in terms of how you can do sub queries, but according to my research on the internet the preferred means for doing this is to use Criteria and DetachedCriteria. Here are some places which I used to come to this conclusion:

The way this works in Hibernate is as follows:

Session session = (Session) getEntityManager().getDelegate();

Criteria crit = session.createCriteria(Foo.class);

// First subquery to get A

DetachedCriteria query1 = DetachedCriteria.forClass(Foo.class);

query1.add(Restrictions.eq("a", 1));

query1.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query1));

// Second subquery to get B

DetachedCriteria query2 = DetachedCriteria.forClass(Foo.class);

query2.add(Restrictions.ge("b", date));

query2.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query2));

// Third subquery to get C

DetachedCriteria query3 = DetachedCriteria.forClass(Foo.class);

query3.add(Restrictions.eq("c", “BAR”));

query3.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query3));

// Fourth subquery to get D

DetachedCriteria query4 = DetachedCriteria.forClass(Foo.class);

query4.add(Restrictions.eq("d", 3));

query4.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query4));

List<Foo> results = crit.list();

Each subquery reads as “get the id’s from Foo where the restriction holds true”, and the criteria represents the combined result of all of the added subqueries.

This isn’t the exact equivalent of the previously specified SQL though. When looking at the HQL debug output it ends up being the following:

SELECT ID, A, B, C, D

FROM Foo

WHERE ID in (SELECT ID FROM Foo WHERE A = 1)

AND ID in (SELECT ID FROM Foo WHERE B >= ‘2011-08-12’)

AND ID in (SELECT ID FROM Foo WHERE C = ‘BAR’)

AND ID in (SELECT ID FROM Foo WHERE D = 3)

This is better than what was there, but it still isn’t as efficient as it could be. In this code we are selecting 4 different times across the entire Foo record set. In the SQL I am trying to write each time we select it is from the previous subset. For example we are looking for condition B inside the A result set,which is smaller than all of the Foo records.

As it turns out, it isn’t that hard. You just add your subqueries to other subqueries like this:

Session session = (Session) getEntityManager().getDelegate();

Criteria crit = session.createCriteria(Foo.class);

// First subquery to get A

DetachedCriteria query1 = DetachedCriteria.forClass(Foo.class);

query1.add(Restrictions.eq("a", 1));

query1.setProjection(Projections.property("id"));

crit.add(Subqueries.propertyIn("id", query1));

// Second subquery to get B

DetachedCriteria query2 = DetachedCriteria.forClass(Foo.class);

query2.add(Restrictions.ge("b", date));

query2.setProjection(Projections.property("id"));

query1.add(Subqueries.propertyIn("id", query2));

// Third subquery to get C

DetachedCriteria query3 = DetachedCriteria.forClass(Foo.class);

query3.add(Restrictions.eq("c", “BAR”));

query3.setProjection(Projections.property("id"));

query2.add(Subqueries.propertyIn("id", query3));

// Fourth subquery to get D

DetachedCriteria query4 = DetachedCriteria.forClass(Foo.class);

query4.add(Restrictions.eq("d", 3));

query4.setProjection(Projections.property("id"));

query3.add(Subqueries.propertyIn("id", query4));

List<Foo> results = crit.list();

This hibernate code results in the desired SQL:

SELECT ID, A, B, C, D FROM FOO fooA WHERE fooA.ID IN (

SELECT ID FROM FOO fooB WHERE fooB.ID IN (

    SELECT ID FROM FOO fooC WHERE fooC.ID IN (

          SELECT ID FROM FOO fooD WHERE D = 3

    ) AND fooC.C = ‘BAR’

) AND fooB.B >= ‘2011-08-12’

) AND fooA.A = 1

Flex 3 Test Automation using QTP

2011-08-11T17:35:00.001-06:00

This information has been sitting in my “to publish” list for a while now, so I am finally going to put it out there since a couple people have been asking about it. It describes my experiences with automating user interface interaction testing using QTP, and all the things I had to workaround.

1. Setting up and running QTP

1.1 Add “automation_charts.swf” to the Flex SDK directory

Example: C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs

1.2 Add the QTP libraries to your project compiler options

Add the following to the compiler options on the project:

-include-libraries "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\automation.swc" "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\automation_agent.swc" "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\automation_charts.swc" "C:\Program Files\Adobe\Flex Builder 3 Plug-in\sdks\3.2.0\frameworks\libs\qtp.swc"

The absolute path must be used, relative will not work.

This will force the libraries into your main application SWF, which should increase its size by about 1.2 MB

1.3 Run your Flex application on a web server.

You must run the application on a web server, it cannot run locally. If you attempt to run it locally you will get lots of browser scripting errors.

Example: http://myserver/foo/bar.html

1.4 You must be running Flash 9

The computer running the application and QTP must have Flash 9, otherwise system popups will be blocked by Flash Player security

1.5 You must run QTP and the browser window in the same monitor.

The computer running the application and QTP must be doing so in the same monitor window in the event that two monitors are being used, more on this later.

2. Required Code Changes

For various reasons when doing any type of user interface testing in Flex you have to make code changes.

2.1 Create new build and deployment

You will need to create a new variation of your build that compiles the automation libraries into the application binary (and all modules). This is because you don’t want to deploy your release binaries with the QTP automation classes compiled into them. This is because it significantly increases the size of your SWF.

2.2 Recursively set the automation hierarchy when running for automation

In order for the QTP to be able to playback button presses to objects deep within the component hierarchy, an automation value has to be set throughout the application at runtime, see “QTP Playback is slow” for information as to why and how.

2.3 Creation of automation delegates for every custom components that QTP does not recognized.

QTP will not recognized interactions with items that inherit directly from UIComponent directly, or are otherwise not a standard button, or text component, or control. See “Custom Automation Delegates” for more information.

3. QTP Specifics and Problems

3.1 Creating a Custom Flex Automation Delegate

References

Summary

Delegates exist so that you do not have to place automation related code in you standard components, after all, most people don’t want to run their applications with automation support. Delegates are provided for all the framework classes and will generally work out of the box for any framework class you extend. The need to write custom delegates arises if you create a custom component that directly extends UIComponent or if you have complex requirements for a component which already has a framework provided delegate.

A delegate has to be created for each custom class. Example:

package

{

import flash.display.DisplayObject;

import mx.core.IInvalidating;

import randomWalkClasses.RandomWalkEvent;

import mx.automation.IAutomationObject;

import mx.automation.AutomationIDPart;

import mx.automation.delegates.core.UIComponentAutomationImpl;

import mx.automation.IAutomationObjectHelper;

import mx.automation.Automation;

import mx.automation.events.AutomationRecordEvent;

import flash.events.Event;

[Mixin]

public class RandomWalkDelegate extends UIComponentAutomationImpl

{

private var walker:RandomWalk

public function RandomWalkDelegate(randomWalk:RandomWalk)

{

super(randomWalk);

randomWalk.addEventListener(RandomWalkEvent.ITEM_CLICK, itemClickHandler);

randomWalk.addEventListener(AutomationRecordEvent.RECORD, labelRecordHandler);

walker = randomWalk;

}

public static function init(obj:DisplayObject):void

{

Automation.registerDelegateClass(RandomWalk, RandomWalkDelegate);

}

private function itemClickHandler(event:RandomWalkEvent):void

{

recordAutomatableEvent(event);

}

public function labelRecordHandler(event:AutomationRecordEvent):void

{

// if the event is not from the owning component reject it.

if(event.replayableEvent.target != uiComponent)

//event.preventDefault(); can also be used.

event.stopImmediatePropagation();

}

override public function get numAutomationChildren():int

{

var numChildren:int = 0;

var renderers:Array = walker.getItemRenderers();

for(var i:int = 0;i< renderers.length;i++)

{

numChildren += renderers[i].length;

}

return numChildren;

}

override public function getAutomationChildAt(index:int):IAutomationObject

{

var numChildren:int = 0;

var renderers:Array = walker.getItemRenderers();

for(var i:int = 0;i < renderers.length;i++)

{

if(index >= numChildren)

{

if(i+1 < renderers.length && (numChildren + renderers[i].length) <= index)

{

numChildren += renderers[i].length;

continue;

}

var subIndex:int = index - numChildren;

var instances:Array = renderers[i];

return (instances[subIndex] as IAutomationObject);

}

return null;

}

override public function createAutomationIDPart(child:IAutomationObject):Object

{

var help:IAutomationObjectHelper = Automation.automationObjectHelper;

return help.helpCreateIDPart(this, child);

}

override public function resolveAutomationIDPart(part:Object):Array

{

var help:IAutomationObjectHelper = Automation.automationObjectHelper;

return help.helpResolveIDPart(this, part as AutomationIDPart);

}

override public function replayAutomatableEvent(event:Event):Boolean

{

var help:IAutomationObjectHelper = Automation.automationObjectHelper;

if (event is RandomWalkEvent)

{

var rEvent:RandomWalkEvent = event as RandomWalkEvent

help.replayClick(rEvent.itemRenderer);

(uiComponent as IInvalidating).validateNow();

return true;

}

else

return super.replayAutomatableEvent(event);

}

The TEAFlexCustom.xml for Mercury on all machines that intend to do automation has to be updated to include details about the custom class. Example:

<TypeInformation xsi:noNamespaceSchemaLocation="ClassesDefintions.xsd"

Priority="0" PackageName="TEA" Load="true" id="Flex" xmlns:xsi="http:/

/www.w3.org/2001/XMLSchema-instance">

<Description>FlexRandomWalk</Description>

<Description>User clicked item</Description>

</Argument>

</Operation>

</TypeInfo>

<Description>To be written.</Description>

</Property>

<Description>The name used by the automation system to identify an object.</Description>

</Property>

<Description>To be written.</Description>

</Property>

<Description>Developer-assigned ID.</Description>

</Property>

<Description>The object's index relative to its parent.</Description>

</Property>

<Description>Number of children open currently</Description>

</Property>

</Properties>

</ClassInfo>

</TypeInformation>

3.2 File Selection fails to playback

When playing back the selection of a file in a System32 file dialog, the following error is thrown in QTP with the message “Object not visible”:

Cause: Dual Monitors
Reference: http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1200371

Summary: This error is sometimes thrown when the browser, the secondary window (the file dialog), and QTP are not all in the same monitor.

Solution: Display all the windows in the same monitor.

3.3 QTP Playback is slow

Cause #1: Bug with automation in Flex 3 SDK

Reference: http://bugs.adobe.com/jira/browse/FLEXENT-749

Summary: QTP playback is slower because of the implementation of automation libraries for Flex in the version 3 SDK. The issue does not exist in the version 2 SDK and has been resolved in the 4 SDK.

This is specifically because deep container hierarchies are slow to iterate through.

Reference: http://raniskumar.wordpress.com/2009/08/26/slow-replay-of-scripts-in-flex3/

In the Flex 3 SDK automation scripts are significantly slower in deep component and container hierarchies. This is because the scripts have to iterate through the entire hierarchy in order to find the component that is being used. This issue was resolved in the Flex 4 SDK,

For example pressing the upload button to open the file dialog took 4 minutes, because the button is so deep within the hierarchy.

Potential Workaround: Set the automationName of the container and component

Reference: http://help.adobe.com/en_US/Flex/4.0/html/WS2db454920e96a9e51e63e3d11c0bf69084-7a4b.html

Workaround Summary

The automationName is a property available every component which designates the name that is used to identify a component in test automation. If no automationName is specified in Flex containers use their id attribute, and if no id attribute is present the container will have an automationName generated for it prefixed with “index”. Buttons however use the value of their label as their automationName when it is not specified.

It was theorized that setting the automationName may allow quicker access to the component, without having to iterate through the entire component container hierarchy, but this did not make a difference.

For example a button without an automationName would be accessed in QTP like the following:

Browser(#).FlexApplication(“Client”).FlexButton(“Upload and Print”).Click

Where “Upload and Print” is the generated automationName from the button label, but if automationName were set on the component in flex to “myButton” then the button would be accessed like the following:

Browser(#).FlexApplication(“Client”).FlexButton(“myButton”).Click

Actual Workaround: Set every container in the application hierarchy to showAutomationInHierarchy to true

References:

Workaround Summary

By default when QTP has to access a Flex component, it has to access that component by automationName. That component is located within an application by iterating starting with the application container and all of its children until the component within that automationName is found. The result is that a component that is deep within a container hierarchy is slow to find. It is for this reason and others, that it is recommend that Flex applications do not overuse container components. Unfortunately in the case of a large already built application, it is too late to go back and redesign the component container structure.

The solution is to specify every automationName within a hierarchy in order to limit the looping needed to find a component. This can be done by setting the showAutomtationInHierarchy value on a container to true, which will cause it to be detected in QTP even if that component is not directly involved in the interaction. If a Flex application uses modules to incrementally load itself over time, it is not possible on startup to recursively set the hierarchy value. Instead after a module is loaded, the module and the components within can recursively have this value set.

public function configureAutomation(obj:DisplayObject) : void

{

if (obj is Container)

{

var c:Container = obj as Container;

c.showInAutomationHierarchy = true;

var children:Array = c.getChildren();

for(var i:int=0; i < children.length; i++)

{

var child:DisplayObject = c.getChildAt(i);

configureAutomation(child);

}

For example without this change the Upload button in the Client is accessed using QTP like the following, which requires 4 minutes to process:

Browser("#").FlexApplication("Client").FlexDividedBox("workflowUploadBox").FlexButton("uploadFirstBtn").Click

When all of the containers have the automation hierarchy set to show, that same button is accessed like the following in QTP and happens instantly:

Browser("#").FlexApplication("Client").FlexContainer("index:29").FlexCanvas("_DocumentViewModule_DocumentVi").FlexDividedBox("workflowUploadBox").FlexBox("index:0").FlexCanvas("newDocView").FlexBox("_NewDocument_VBox1").FlexCanvas("uploadView").FlexBox("_UploadView_HBox2").FlexButton("uploadFirstBtn").Click

Cause #2: QTP 9.1 is slow

Summary: Script playback in QTP 9.1 is very slow, and switching to 9.5 makes the playback significantly faster.

4. Maintainability Concerns

This is a concern with Flex and HTML based user interface drivers, in which the actor has to know the ID of the component in order to play back some action on it. In languages like JSF and Flex you don’t have to give an XML component an ID, which results in that component getting a dynamically generated ID.

In Flex the problem is that when a component is not explicitly named, that name is calculated at compile time. The problem with the generated name is that it is based on a component’s position in its hierarchy. For example when a new Canvas is added, the names of the other two canvases change as shown by the highlighting in red.

It should also be noted that buttons are labeled by their text, which means of the text on the button changes the script will also have to change.

In order to ensure that a component’s name to QTP is always the same, the developer has to set the component’s automation name in the code for buttons and labels, and the component’s ID for boxes and canvases.

Building a Custom 2D Map Component from Scratch using Virtual Earth

2010-09-01T15:55:00.003-06:00

References

Bing Maps Tile System (http://msdn.microsoft.com/en-us/library/bb259689.aspx) - Describes the tiling system and gives C# code for converting between coordinate systems.
Virtual Earth Deep Zooming (http://www.silverlightshow.net/items/Virtual-earth-deep-zooming.aspx) - Among other things it describes the URL format for tiles.
Calculate distance, bearing and more between two Latitude/Longitude points (http://www.movable-type.co.uk/scripts/latlong.html)
Geographic coordinate system (http://en.wikipedia.org/wiki/Geographic_coordinate_system) - Expressing latitude and longitude as linear units
Ricky's Bing Map Blog (http://rbrundritt.spaces.live.com/) - Almost everything you would ever need to know about working with Virtual Earth with examples and math.

Why? Because in the technology you are using a map component does not exist, or you are unable to use the existing map component for some reason.

Coordinate Systems There are several coordinate systems involved with using a tile service like Virtual Earth. Virtual Earth and other tile services divide the world in to fixed size tiles at different zoom levels denoted by z, where the number total number of tiles is 2^2z. The map in this example is at zoom level 3 so there are 64 tiles total, and is derived from the images in Virtual Earth article in the Microsoft Bing Help Docs (http://msdn.microsoft.com/en-us/library/bb259689.aspx).

The world geographic coordinate system (represented in black) is in latitude and longitude and runs from approximately 85 to -85 latitudes and -180 to 180 longitudes. Due to the way the earth in compressed in a tiled map, the lines of latitude are now evenly spaced even through they are the same pixel distance apart. This requires a formula to be used to determining latitude that takes into account this difference.

The tile coordinate system (represented in red) is in row and column, and describes the position of tiles on the world map. All tiles are of a fixed size, though typically at a size of 256 pixels as in the example map, and run from top left to bottom right. Positions in between the upper left corners of tiles are represented using decimal values, so for example (1.5, 0.5) would be in the exact center of tile (1.0, 0.0).

The world pixel coordinate system (represented in blue) is in x and y, and runs from top left to bottom right. It is used to describe the tile coordinate system where tile (0, 0) corresponds to pixel (0, 0). The map size in pixels is t_s(2^z),where the tile size is denoted as t_s and is 256 while z denotes the zoom level and is 3 in the case of the example map.

The screen pixel coordinate system (represented in green) is in x and y, and describes the position of the screen (actually the component displaying the map) in relation to the world map. Because at most zoom levels the world map in pixels is larger than any monitor can display, only the portion of the map which can be seen within the map component is rendered. This means that the size of the component being displayed is dynamic and dependent on the specific application’s usage, as well as how the local x and y relates to the global x and y of the world pixel coordinate system.

General Formulas and Notations

The following is a list of several commonly used formulas and terms.

Latitude and longitude to row by column

This formula is used to turn a latitude and longitude form the world geographic coordinate system into a and row by column value in the tile coordinate system.

World x and y to latitude and longitude

This formula is used to convert and x and y position in the world pixel coordinate system to a latitude and longitude in the world geographic coordinate system.

Requesting Tiles

Tiles are requested based on retrieving them one at a time from a URL. How a URL is generated depends on the particular tile server and in the case of this map component only the Virtual Earth tile servers are used. The URL is used to specify the type of image, its zoom and location, and its file type. All files types in Virtual Earth are of PNG or JPG, and the location must be given using the quad key. The quad key is a compressed representation of the tile location using the row, column, and zoom.

The following shows how to generate a quad key using the row, column, and zoom using a Java-like syntax:

String tileToQuadKey(int r, int c, int z) {

String quadKey = "";

for (int i = z; i > 0; i—) {

char digit = '0'; int mask = 1 << (i - 1);

if ((c & mask) != 0)

digit++;

if ((r & mask) != 0) {

digit++; digit++;

}

quadKey += (digit + "");

}

return quadKey;

}

Tile Request and Display Optimization

Only the tiles which are within view of the screen (component) need to be displayed, which means that those should be the only ones to be requested. Once a tile is displayed in order to cut down on loading and rendering time the map component stores the tile, and when it is no longer in view keeps the tile in memory but removes it from the display. For example if the map is to be centered at the tile location of (3.5, 3.5) and is component displaying the map is only large enough to display half of each surrounding tile, than only 9 tiles need to be requested.

In the event that the screen moves up into the next row of tiles the tiles that are now in view should be requested and displayed, while the three tiles that are no longer displayed should be cached.

In order to know which tiles to display and where a center coordinate must be known. Given that center coordinate and the size of the area that will display the map, the tiles to be displayed and their position within that component can be determined.

Determining the tiles to be displayed in the current screen using a center coordinate

Given a coordinate in latitude and longitude to be the center on the map to be displayed, that coordinate can be converted to a tile location in row by column. By using that row and column the range of tiles to be retrieved and displayed can be determined using the amount of space available on the screen and the tile size. The row and column minimums and maximums denote the range of tiles to be displayed, and the delta x and y are used to calculate the pixel distance from the center of the center tile to its upper left corner. This is used to determine how many tiles can fit above, below, to the right of, and to the left of the center tile according to its current position in the center of the screen.

For example if the center coordinate translated to (c, r) = (2.5, 4.5 ) at zoom level 3 and the screen was 512 by 512 pixels, then the column and row minimums and maximums would be calculated as approximately the tiles needed to cover the size of the current screen.

Positioning tiles relative to center

With the range or rows and columns for all tiles to display, given the row and column of a tile its local position in x,y is calculated in the following way:

The x,y local of the tile of the given row and column is calculated relative to the position of the center image. This is done so that panning movement can be calculated relative to the center position of the screen, in order to calculate a simple offset that can be applied to all tiles in order to handle map panning. For example if the center tile location was (c,r) = (2.5, 4.5) and the screen was 512 by 512 pixels, the position of the tile at (c,r) = (1, 3) relative to the center tile would be at (x,y) = (-256, -256).

Handling Map Panning

The end result is that the user should be able to click and drag the map in order to move it around (or to pan). This requires keeping track of where the mouse was clicked, where it was dragged to, and when it was released (to stop worrying about where it is being dragged). During a drag the difference between the current mouse location and the location of the original click can be calculated, and then used to determine the new offset to apply to the map along with the new visible rows and columns.

The variables such as the x,y and r,c min and max values need to be kept track of through the map component life cycle. This is because positioning is done using an offset based on the starting location of the center tile. The offset is calculated using the inverse of the change in mouse position during the click and drag. For example if the user clicks on the map and drags right, more of the map needs to be proportionally drawn to the left. The x and y min and max are used to track the size of the tile area in pixels, in order to determine the how many more tiles are needed at the new map boundaries in terms of rows and columns.

The following shows how the variables in the map panning equation are changed when the screen view (512 by 512 pixels) is moved up on the map by 256 pixels:

Zooming in on a Set of Locations

A function available within most maps is the ability for the map to determine the zoom level to best fit a series of points, and center appropriately. This is generally done by determining the area required in order to display a number of points, and determining the closest matching meters/pixels zoom level and then centering appropriately.

Determining the geographic display area

The geographic display area is determined by taking the minimum and maximum latitudes and longitudes of the list of geographic position, which can be used to construct the geographic positions denoted as A,B, and C. Note that this method for determining geographic bounds will not work across the corners of the map.

Calculating the distance between two geographic points

After determining the geographic display area that is required in terms of the rectangle ABC, the distance from geographic coordinates A to B and B to C needs to be calculated. This requires the ability to calculate the distance between two geographic coordinates, which is described in the following as the function d(A, B):

Calculating the Desired Meters per Pixel

With the the ability to calculate the distance between geographic coordinates, the size of the geographic area to be zoomed in on can be calculated by determining the meters per pixel required to cover the area. This formula assumes a fairly square screen, because it takes the largest distance and meters against the smallest pixel length of the required screen area.

For example using the distances AB and BC, since AB is the largest it is used to calculate the meters per pixels required to display all of the points within the area of rectangle ABC with center G.

Finding Zoom Level based on Meters Per Pixel

(Table from http://msdn.microsoft.com/en-us/library/bb259689.aspx)

A Flex Ant Build: Optimized Modules, Libraries, Runtime CSS, HTML Wrapper, Runtime Images

2010-03-24T18:14:00.011-06:00

A common need in a system involving Flex is for the ability to build the Flex portion using Ant. While there are some online examples, there is not one that I have been able to find that does and describes all of the things I typically need to do in Flex . The things which I typically need to do are the following:

Build optimized modules without having to declare a separate compile target for every module
Include one or more SWC libraries
Compile the CSS into a SWF to be loaded at runtime
Create a custom HTML wrapper, that is also responsible for setting the application endpoint
Including images to be loaded at runtime
Uses a properties file to define all the elements

This build also assumes the following:

There is one source directory called "src"
Runtime images are somewhere under the source directory
CSS is somewhere under the source directory
All modules are in a single directory, that contains nothing but the modules
You have the Flex ant task JAR in libs
You have the Ant Contrib JAR in libs
You have a separate index.template.html for the build that contains ${} variables for the endpoint

For example the following is my example project directory structure:

The build works by executing these targets as part of its default target:

clean - deletes the deployment directory
compile - compiles the main application into its SWF
compile-modules - uses a "for" loop on all the MXML files in the modules directory and calls the "compile-module" target on them
compile-module - compiles a module into a SWF file, optimized for the main application
compile-css - compiles the CSS into a SWF
compile-wrapper - builds the HTML wrapper for the application, and includes the endpoint using FlashVars
copy-images - Copies runtime images from source into deployment
clean-up - deletes any generated files

Here are the build files:

Dear John, what is the easiest way to use Flex modules?

2010-02-09T22:35:00.004-06:00

Question asked by Ezra.

When it comes to modules there are two general schools of practice:

Load and cache
Load and unload

Option 1 is by far the most simple, but since content is never unloaded the application has to be small enough to theoretically be loaded in its entirety. I tend to build most Flex applications under this model.

Option 2 is near impossible depending on the specific implementation. There are many areas beyond what I have written about that can prevent a module from from unloading. I have built applications under this model, and getting things to unload correctly is a massive undertaking.

Implementing option 1 can be as simple as putting a view stack in your main application file and using states to add ModuleLoaders to it, and as complicated as writing your own module manager utility to load modules based on some external configuration.

An important concept with modules that you must always consider though is compile by reference. When compiling a Flex application into a SWF file, it is that class which inherits from “mx.core.Application” and the referenced classes within its class hierarchy that are compiled into that SWF file along with other embedded resources. Classes that are not referenced in that class hierarchy are not compiled into the application SWF file. This concept of compilation by reference in a Flex application also applies to a Flex module.

Knowledge of compilation by reference and their effects on modules and applications is important, because if an application has reference to a module class in its class hierarchy that module class is compiled into both the module SWF file and application SWF file. This duplication of module class definitions is undesired behavior, since the main purpose of modules is to have code and resources in a location outside of the application SWF file and not within it. Beware that the several examples I have seen in Adobe documentation and elsewhere show the main application give direct reference to the module class that it is trying to load. This results in that module class being compiled into both the main application SWF and the module SWF, which defeats the purpose of using modules.

Modules are also effected greatly by optimization by eliminating class duplication. For example if your main application, ModuleA, and ModuleB reference FOO, if ModuleA and ModuleB are optimized for that application the class definition for FOO is compiled into the main application SWF file. If those modules were not optimized then the class definition for FOO would be compiled into the main application, ModuleA, and ModuleB SWF files.

All of this means that your modules should be optimized for your specific application, and that your application should not directly reference modules and your modules should not directly reference your application. The following is some pseudo-code that shows how to use states and modules loaders to create a modules application:

This example requires that both a ModuleA.mxml and ModuleB.mxml exist as Flex modules, which results in them getting compiled into their own SWF files. In order to interact with the application I typically create a singleton business object that maintains the current state of the application, shared data, and has the ability to change state.

For example:

[Bindable]

public class AppStateBO {

public static const STATE_A:String = "STATE_A";

public static const STATE_B:String = "STATE_B";

public var currentState:String = "";

//singleton stuff here

}

That why both the main application and each module can have access to the data they need to share and the ability to change the state of the application:

[Bindable]

var appState:AppStateBO = AppStateBO.getInstance();

...

appState.currentState = AppStateBO.STATE_A;

If the currentState attribute of the main application were bound to appState.currentState then any module with the app state singleton could change the state. There are of course many other ways to do this, but at a minimum you need something to be shared across the app and modules.

Dear John, How do I remove Flex bindings on an object?

2010-02-09T22:12:00.004-06:00

Question asked by Joshua.

Removing bindings requires a trick with include namespace mx_internal, which will give you access to parts of components that you would normally not have access to.

According to the source code for Binding.as (which I believe is not included in the SDK), the bindings are stored in the following places within objects:

mx_internal _bindings:Array
mx_internal _watchers:Array
mx_internal _bindingsByDefinitions
mx_internal _bindingsBeginWithWord

The following function can then be used to clear bindings on an object:

Dear John, How do I unload Flex modules?

2010-02-09T21:58:00.003-06:00

Question asked by Joshua.

Unloading modules is a difficult thing to do, and is near to impossible in Flash Player 9. Originally this article clued my into what was going on: http://www.gskinner.com/blog/archives/2008/04/failure_to_unlo.html. Essentially to get modules to truly unload you have to get rid of all the references to them, and be using the "Copied Domain Method." The copied domain method is described here in detail, along with its effects on memory: http://jvalentino.blogspot.com/2009/06/flex-memory-issue-4-modules-and.html.

Assuming you are using the Copied Domain Method, you then have to deal with the following areas of leakage as mentioned in the originating article:

State and AddChild – This can cause a leak, sort of, see http://www.nbilyk.com/flex-states-memory-leak
Bindings – Bindings can sometime leak, see http://bugs.adobe.com/jira/browse/SDK-14875
Forcing Garbage Collection – You have to force GC most of the time to get modules to move along, see http://jvalentino.blogspot.com/2009/05/flex-memory-issue-3-garbage-collection.html
Listeners – You have to remove listeners when you are done with them.
Using the Current Domain Method – prevents modules from truly unloading, see http://jvalentino.blogspot.com/2009/06/flex-memory-issue-4-modules-and.html
Effects – Effects in a module will cause it to leak, see http://jvalentino.blogspot.com/2009/05/flex-memory-issue-2-component-effects.html

I essentially handled these area of leakages by iterating recursively through every component within a module class after unloading, removing state, removing bindings, removing common listeners, reseting the EffectManager, and then forcing garbage collection using what is called the local connection hack.

The problem with this though is that there will always be memory accumulation even if references are removed. This is due to a largely unknown and undocumented issue known as "Rendering memory fragmentation." I have discussed this issue with several engineers at Adobe through a client which I cannot name, and they are at least passively aware of it but it is an issue with the Flash Player itself. The article at http://www.craftymind.com/2008/04/09/kick-starting-the-garbage-collector-in-actionscript-3-with-air/ hinted at what was going on, which lead me into a full investigation to verify the problem.

Basically the Flash Player operates on what is called an Elastic Racetrack (http://www.onflex.org/ted/2005/07/flash-player-mental-model-elastic.php), which ends up with the memory used of AS3 object allocation and the memory used for rendering getting on the same page in memory. When the garbage collection runs the allocation space is not cleaned because the blocks contain active rendering memory and vice versa. This makes it to where the garbage collector fails to clean things, which causes leaks. Since the issue is rendering related it is more prevalent in application that are graphically intense and run at large resolutions, like kiosks. It is for these reasons that I recommend that most applications pool modules, so that the rendering and object instantiations through allocations are minimal.

In the end it will depend on how many "modules" are in your application, how big they are, and if they are really modules. It is quite common for application to incorrectly use modules because they followed the examples released by Adobe, which cause the module definitions themselves to be compiled into both the module SWF and the main application SWF. Flex operates on a compile by reference basis, so if your application has direct reference to a module class that module class gets compiled into both the module SWF and the main application SWF.

Flex: Building Modular Rich Internet Applications

2009-07-18T14:38:00.001-06:00

Why? Because you want to incrementally deliver content

A typical model for building a Flex application is to just use one SWF file. That is because by default when you create a Flex project in Flex Builder your main application MXML and all of the classes that it references are compiled into a single SWF file.

Flex Builder then embeds that SWF file in an HTML page where you can then upload it and the Flex content somewhere, and when people visit that page it and the SWF file are downloaded by their browser and Flex application is executed.

It helps to conceptualize as an application in terms of its views or screens. For example an application may consist of a screen to login, a screen to create an account, and so on. In the single SWF model that single file contains all of these different views. You begin to get into trouble when you have a lot of different views, which causes the single SWF file to become large.

For example you could have an application that has 10 different views where each view adds about 200 KB to the application size, the result is that you end up with a 1.95 MB file that the user has to download at one time to run it. This of course would be come a problem when you have a lot more views, views that are larger, or a combination of both.

Flex has the ability to use Modules, which allows you to break an application up into parts. Typically one would break an application up into its views (or screens). The result is an application that the end user downloads piece by piece and as needed. For large applications this is a requirement, since users don’t want to have to wait for several minutes of downloading every time they need to use your application.

How?

I really haven’t been ever able to find a definitive source that explains all of the different ways to use modules, which is why I am writing this. When it comes to modules there are two general approaches:

The text-book approach – This is the way modules are described in documentation and other Adobe writings, which does not take into account real world functional uses problems.
The pragmatic approach – This is the way they are actually used in functioning applications, taking into account the plethora of bugs.

I will attempt to explain both approaches, with more emphasis on the pragmatic since a working application accompanies this writing.

Modules and Application Domains

Application Domains in the case of modules are used to determine what parts of a module are actually unloaded when they are unloaded. The following are the two methods in which this functionality is controlled:

Current Domain Method – This makes it do modules never really unload, and that their definitions are kept around for the lifetime of the application. The result is modules load faster the second time they are loaded because they never really unloaded in the first place.

Copied Domain Method – This is the default behavior and is supposed to allow modules to truly unload. The result is that modules always take the same amount of time to after unload since they have to be completely reloaded.

For more information see the following: http://jvalentino.blogspot.com/2009/06/flex-memory-issue-4-modules-and.html

The Realities of Modules

Basically looking at a module wrong will cause it not to unload when using the Copied Domain (Default) Method. The following are culprits for the modules persisting, each of which I have implemented resolutions:

State and AddChild – This can cause a leak, sort of, see http://www.nbilyk.com/flex-states-memory-leak
Bindings – Bindings can sometime leak, see http://bugs.adobe.com/jira/browse/SDK-14875
Forcing Garbage Collection – You have to force GC most of the time to get modules to move along, see http://jvalentino.blogspot.com/2009/05/flex-memory-issue-3-garbage-collection.html
Listeners – You have to remove listeners when you are done with them.
Using the Current Domain Method – prevents modules from truly unloading, see http://jvalentino.blogspot.com/2009/06/flex-memory-issue-4-modules-and.html
Effects – Effects in a module will cause it to leak, see http://jvalentino.blogspot.com/2009/05/flex-memory-issue-2-component-effects.html

Even with these resolutions there is ultimately a problem that is an article unto itself regarding how the memory required to render components becomes fragmented over time, eventually preventing objects that should be eligible for collection from by collected. I will get around writing about it and the proof of its existence later, but for right now just be aware of its effects. Its effects are most noticeable in Flex applications that run at high resolutions, because the larger in size the component the more memory that is required for rendering it.

This basically means that in larger and/or graphically rich applications if you have to render a lot of content over and over you will run out of memory and there is nothing you can do about it. Using the current domain method will mitigate this problem, but then you have to be sure that your application’s definitions can fit within whatever the requirement is for client memory. Even then you will still encounter the problem within rendering memory fragmentation.

What can be done?

Pool your modules.

Once a module has been created store its instance and then retrieve it the next time it is needed. This however requires enough memory available to be able to go to every module in the application, or however modules it is possible to go to.

The result is a memory profile that looks like the following with modules in which the instance never go away:

Using this method you can load modules over and over again without having to worry about the issue with rendering memory fragmentation (unless there is a substantial amount of dynamic manually drawn content).

The resulting secondary load times are also much faster than when using the Current Domain Method:

What are my options?

Pool your modules or don’t – If you pool your modules then you need to have enough memory available for the possibility that the user loads a lot of modules. Otherwise you are going to hit the problem with rendering memory fragmentation and run out of memory.
Use the Current Application Domain or the Copied Application Domain Method – If you use the copied domain method it will be extremely difficult to get modules to actually unload, and you will eventually run out of memory in most applications due to the problem with rendering memory fragmentation. If you use the current domain method you will mitigate the the rendering memory fragmentation problem, but you will need to be able to have enough space in memory to deal with the possibility of having to store the definitions for a large number of modules.
Force Garbage Collection or don’t – If you don’t force garbage collection and you are not pooling modules, then modules sometimes won’t unload. When pooling it will help memory along, but in both cases be aware then forcing garbage collection can cause problems with remoting.
Cleanup on module unload or don’t – If you don’t “cleanup” when not using pooling then various instances within modules are likely to leak.

How are all these options implemented?

The following is a test application featuring a ModuleManagerUtil class and a MemoryManager class.

Application: http://modular.theflexsite.net/

Please be aware that the debug version of the Flash Player always leaks modules, so if you are using the debug player just about everything is going to leak. See the following for more information: http://jvalentino.blogspot.com/2009/05/flex-memory-tip-1-debug-player-always.html

About the Sample Application

The ModuleManagerUtil class is used to handle loading and pooling modules.

The MemoryManager class is used to get the current amount of memory in use by the Flash Player and force garbage collection.

The buttons on the left are used to load different modules. The simple modules just contain panels, the complex modules contain DataGrids, and the popup module will load a module as a popup.

The bottom of the page is a JavaScript plotting application done using Flot, so display memory without interfering with what is going on in the Flex application.

Flex + BlazeDS + Google App Engine + Java + Eclipse

2009-06-22T21:45:00.001-06:00

Why?

Because you want to write a Flex application that requires that the server be able to push data to the client, you want a highly scalable server so you decided on the Google App Engine, when provided with the option of Python versus Java you found Java more accessible, and Eclipse is an easy way to link it all together.

Flex – used for the graphical user interface
BlazeDS – used for Flex to Server remote objects and provides the ability for the server to push data to the Flex client
Google App Engine – used for hosting the server portion of the application
Java – the language used for constructing the server potion of the application
Eclipse – an easy way to build, manage, write code, and deploy

There are lots of other options though, in general with Flex you are aiming for a way to deal with remote objects and/or server data push + some way to persist data on a server that ties to some language + some IDE. This is just one of many possible configurations with an emphasis on relatively free parts.

It is also apparently difficult to get all of things working together, so I thought it would help for others if I shared the steps.

In this example I basically combined the two following projects into one to use the Google App Engine to show basic remote objects and messaging:

If you want more details into the how remote objects and messaging works see those examples.

Requirements

Eclipse - http://www.eclipse.org/europa/ (FREE)
Google App Engine Developer Account - http://appengine.google.com/ (FREE)
Install the Google App Engine Java plug-in for Eclipse - http://code.google.com/appengine/docs/java/tools/eclipse.html (FREE)
Flex Builder plug-in for Eclipse - http://www.adobe.com/products/flex/features/flex_builder/ ($$$)
The BlazeDS Binary Distribution- http://opensource.adobe.com/wiki/display/blazeds/Release+Builds (FREE)

1. Creating the Server

1.1 Press the “New Web Application” button in the Eclipse toolbar

1.2 In the “New Web Application Project” window specify a project name, the package, and for the rest use the default settings then press the “Finish” button.

1.3 If you are like me and already run another server on port 8080, then you need to change the project settings to use a different port. You can do this by opening the debug menu and selecting “Open Debug Dialog…”

1.4 Specify a new port to be used and save these settings.

2. Adding BlazeDS to the Server

2.1 Assuming you have obtained the BlazeDS WAR file, unzip it using some ZIP program

2.2 Open the BlazeDS web.xml file and copy the mappings into the web.xml in your Google test project.

The web.xml file is located in MyGoogleServer/war/WEB-INF

The resulting web.xml file should look like the following:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE web-app
PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
"http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>

<welcome-file-list>
    <welcome-file>MyGoogleServer.html</welcome-file>
</welcome-file-list>

<servlet>
    <servlet-name>greetServlet</servlet-name>
    <servlet-class>com.example.myproject.server.GreetingServiceImpl</servlet-class>
</servlet>
<servlet-mapping>
    <servlet-name>greetServlet</servlet-name>
    <url-pattern>/mygoogleserver/greet</url-pattern>
</servlet-mapping>


    <listener>
        <listener-class>flex.messaging.HttpFlexSession</listener-class>
    </listener>

<servlet>
        <servlet-name>MessageBrokerServlet</servlet-name>
        <display-name>MessageBrokerServlet</display-name>
        <servlet-class>flex.messaging.MessageBrokerServlet</servlet-class>
        <init-param>
            <param-name>services.configuration.file</param-name>
            <param-value>/WEB-INF/flex/services-config.xml</param-value>
        </init-param>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>MessageBrokerServlet</servlet-name>
        <url-pattern>/messagebroker/*</url-pattern>
    </servlet-mapping>

</web-app>

2.3 Copy the BlazeDS flex directory into MyGoogleServer/war/WEB-INF

2.4 Open the services-config.xml file and change the three instances of {server.name}:{server.port}/{context.root} to your server name, which in the case of this example is localhost:9000. When you deploy this thing to Google you will of course need to change it.

2.5 Open the appengine-web.xml file and add the node <sessions-enabled>true</sessions-enabled>

2.6 Copy all of the JAR file in the BlaseDS lib directory to your lib directory, which in this example is MyGoogleServer/war/WEB-INF/lib

Before copy:

After copy:

2.7 If your server will be at a different location than your client, then you need to create a crossdomain.xml file to grant the client access.

In this case I like to share, so I am granting the entire internet access to the web services here. Normally you wouldn’t want to do this for reasons that should be obvious:

<?xml version="1.0"?>
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*" />
</cross-domain-policy>

3. Creating server-side logic

3.1 Create a class that will be used as a remote object that will just return “Hello”

package com.example.myproject.hello;

public class Hello {
    public String sayHello() {
        return "Hello World";
    }
}

3.2 Open the remoting-config.xml file and add the following node to map the Hello remote object:

<destination id="Hello">
        <properties>
            <source>com.example.myproject.hello.Hello</source>
        </properties>
</destination>

3.3 Open the messaging-config.xml file and add the following node to be able to use the chat variable for messaging:

4. Creating the Flex client

4.1 Run the Google Server project

4.2 From the File menu create a new Flex Project

4.3 Specify the project name and set it to be a J2EE application server and don’t use WTP.

4.4 Uncheck the box to use the default LiceCycle Data services server and specify the location of the war directory of the Google Server project, and change the URL to reflect the the location of the server. Since I am planning on running the client on a different machine than the server, specify the Output folder to be bin-debug.

4.5 Specify the output folder URL to match the location on your local test server.

4.6 Run two instance of the Flex application, and notice how the sayHello button tests the Hello.java class, and you can use the Send B button to send message back and forth the the “chat” variable.

4.7 The Flex Code

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" creationComplete="consumer.subscribe()">
    <mx:RemoteObject id="blazeService" fault="faultHandler(event)" source="hello.Hello" destination="Hello">
        <mx:method name="sayHello" result="resultHandler(event)" />
    </mx:RemoteObject>
     <mx:Consumer id="consumer" destination="chat" message="messageHandler(event.message)"/>
    <mx:Producer id="producer" destination="chat"/>

    <mx:Script>
        <![CDATA[
            import mx.rpc.events.ResultEvent;
            import mx.rpc.events.FaultEvent;
            import mx.messaging.messages.AsyncMessage;
            import mx.messaging.messages.IMessage;
            private function send():void{
                var message:IMessage = new AsyncMessage();
                message.body.chatMessage = msg.text;
                producer.send(message);
                msg.text = "";
            }
            private function messageHandler(message:IMessage):void{
                log.text += message.body.chatMessage + "\n";
            }

            private function faultHandler(fault:FaultEvent):void {
                result_text.text = "code:\n" + fault.fault.faultCode + "\n\nMessage:\n" + fault.fault.faultString + "\n\nDetail:\n" + fault.fault.faultDetail;
            }
            private function resultHandler(evt:ResultEvent):void {
                result_text.text = evt.message.body.toString();
            }
        ]]>
    </mx:Script>
    <mx:Button x="10" y="132" label="sayHello" width="79" click="blazeService.getOperation('sayHello').send();"/>
    <mx:TextArea x="10" y="27" width="319" height="100" id="result_text"/>
    <mx:Panel title="Chat" width="100%" height="100%">
        <mx:TextArea id="log" width="100%" height="100%"/>
        <mx:ControlBar>
            <mx:TextInput id="msg" width="100%" enter="send()"/>
            <mx:Button label="Send B" click="send()"/>
        </mx:ControlBar>
    </mx:Panel>

</mx:Application>

5. Last Thoughts

I haven’t tried it, but apparently if you want to get remoting-only BlazeDS to work you have to do the following hack: http://martinzoldano.blogspot.com/2009/04/appengine-adobe-blazeds-fix.html

Flex Memory Issue #4: Modules and Application Domains

2009-06-12T22:20:00.001-06:00

“ApplicationDomains are pretty much a neverending source of confusion and suffering…”

- Roger Gonzales, helped in the design of Application Domains in Flex

If you are using modules in Flex whether you are aware of it or not, you are using Application Domains. In particular I have had several issues with them and their effects on memory. An Application Domain is a container for discrete groups of classes, which are used to partition classes that are in the same security domain.

What? Yes, I know. That definition is from Adobe’s asdoc for the ApplicationDomain class, and it really doesn't tell you what it has to do with modules. For a start Roger Gonzales did a great job of explaining the history and purpose of the Application Domain on his blog, located at http://blogs.adobe.com/rgonzalez/2006/06/applicationdomain.html.

To attempt to summarize the Application Domain it is essentially a way to contain AS3 definitions, which effects how modules load and unload. With modules Application Domains let you do one of two things, which I refer to as the “Current Domain Method” and the “Copied Domain Method”

Current Domain Method

Example Usages:

var info:IModuleInfo = ModuleManager.getModule("com/foo/FooModule.swf");
info.load(ApplicationDomain.currentDomain);

This loads a module into the current Application Domain, so that when it unloads the class definition stays in memory but the rest of the module related instances are supposed to go. The next time the module is loaded the class definitions are already there so it loads faster.

For example if you have an Application that has 3 modules and you load one module at a time using the Current Domain Method, unload them, and load the next module in the sequence you get the following memory profile assuming you force garbage collection (http://jvalentino.blogspot.com/2009/05/flex-memory-issue-3-garbage-collection.html) after each unload:

* Assumes modules are all the same size, garbage collection is forced after unload, memory snapshot is taken using System.totalMemory, and memory snapshot is taken between unload of previous and load of new

Notice that the memory peaks after all of the modules are loaded, levels off when those three modules are again loaded, and then only drops down slightly after all the modules are unloaded. The remaining memory isn’t just Flash Player stuff, it is also the definitions for the modules classes. This means that when using the Current Domain Method, modules will never truly unload.

For another example if you look at the time it takes those module to load for the 3 module scenario using the Current Domain Method, you get the following:

The initial load times for each module are much greater than the amount of time for next time the same module is loaded. This is because the definitions for those modules are stored in the current Application Domain for the application when using the Current Domain Method.

Theoretically, when would one use the Current Domain Method?

You would want to use the current domain method if you were not worried about the user loading enough modules to cause the application to go out of memory. When you are using this method you are essentially caching the module definitions when you load them, allowing for the module to be loaded much faster the next time it is needed. To summarize it requires more memory in order to achieve faster load times.

Copied Domain Method

This is what happens when you use the load function of the ModuleManager and specify the parameter as a new Application Domain using the current Application Domain. This is also the default behavior when not using a parameter at all.

Example Usage:

var info:IModuleInfo = ModuleManager.getModule("com/foo/FooModule.swf");
info.load();

or…

var info:IModuleInfo = ModuleManager.getModule("com/foo/FooModule.swf");
info.load(new ApplicationDomain(ApplicationDomain.currentDomain));

This loads the module into a new Application Domain so that they are no references to keep module definitions from unloading. The result is that when you unload a module all of the related classes are supposed to entirely unload. The amount of time required to load a module never changes, because it has to be completely loaded each time.

For example if you have an Application that has 3 modules and you load one module at a time using the Copied Domain Method, unload them, and load the next module in the sequence you get the following memory profile assuming you force garbage collection (http://jvalentino.blogspot.com/2009/05/flex-memory-issue-3-garbage-collection.html) after each unload:

Notice that when the all modules are unloaded the memory is supposed to go back down to where it was before the first module was loaded.

For another example if you look at the time it takes those module to load for the 3 module scenario using the Copied Domain Method, you get the following:

The amount of time that it takes to load each module never changes, because each time the module has to be completely loaded. It is my understanding that the reason for this is because when using the Copied Domain Method you are not adding AS3 definitions to the current domain, so those definitions can fully unload when the module is unloaded.

Theoretically, when would one use the Copied Domain Method?

You want want to use this method if your application was under some kind of memory constraint, or if there were so many modules that it could cause the Flash VM to run out of memory. The down side to this is that you have to fully load each module every time you need it, whether it has already been loaded or not.

Current Domain Method versus Copied Domain Method in the real world

In large and stylistically complex applications in the real world, the copied domain method has provided me with a world of trouble when it comes to memory. The memory behavior is erratic to say the least in the following ways:

Even when forcing garbage collection the memory jumps around erratically like nothing is being done at all.
If you run the same exact scenario over and over again you get different results. One time memory averages 30 MB, another 50 MB, and another time it just runs out of memory

I know what you are thinking, “there are other memory leaks in the application!” That would be incorrect, as the profiler shows no leaking and if you change to the Current Domain Method the problem is gone in the particular scenario in which this happens. It is that point which I became completely confused and I was even able to talk to Adobe about it through a third party, but I was never able to get an answer.

The scenario in particular was the loading and unloading of two different modules over and over again about 100 times. Load Module 1, unload Module 1 and load Module 2, unload Module 2 and load Module 1, etc.

When using the Copied Domain Method each time I ran this scenario I got different results:

For example the first attempt at this scenario was fine, the second attempt ran out of memory, and the third attempt went up in memory and never came back down.

If you take that same exact scenario and make it use the Current Domain Method you get the following:

Using the Current Domain Method seemed to have fixed the memory issue in this scenario, but by using this method I was stuck with modules with definitions that never get unloaded.

Flex Memory Issue #3: Garbage Collection

2009-05-28T18:48:00.001-06:00

Garbage collection is an in depth subject when it comes to modern programming languages. Wikipedia provides a general statement that summarizes what it is:

“…garbage collection (GC) is a form of automatic memory management. The garbage collector, or just collector, attempts to reclaim garbage, or memory used by objects that will never be accessed or mutated again by the application.“ http://en.wikipedia.org/wiki/Garbage_collection_(computer_science)

The expectation of a programmer in a language like Flex or Java, is that when widget X is no longer being used in the application that it gets removed from memory. That it is pretty simple scenario that is unrealistic, considering all graphical user interface components maintain references to other components which contain references to other components and so on and so forth. Flex also adds a few more layers of complexity by including modules and binding.

Flash Garbage Collection Basics

There is a must-read presentation on an atomic view of Flash Player garbage collection written by Alex Harui sometime in 2005, from which I obtained a lot of my high level information. Somehow I located the presentation through the internet and downloaded it, but I have since lost the link.

Flex garbage collection is synonymous with Flash garbage collection, since the memory being talked about is that of the Flash Player. Flash memory allocation basically works by grabbing chunks of memory from the Operating System, and carving those chunks when needed into smaller blocks of fixed size that together as a group are called a pool. When a pool is used up then another chunk is taken from the OS and becomes another pool.

A block of memory can be used, freed, or unused. Used means that there is something in there, freed means that something is still in there but it no longer used and can be garbage collected, and unused means that the block was freed and had been set to unused by the garbage collector.

This means that Flash garbage collection is based on allocation since more memory is not obtained from the OS until Flash needs some for itself but is out. Before Flash decides that another pool is needed though, it might attempt to run garbage collection. The garbage collector will move through Flash memory looking for freed blocks that can be set to unused.

There are also a few important things to note about garbage collection when looking at its effect on memory:

1. You generally can’t predict when garbage collection will run by itself

2. Garbage collection is based on allocation, so memory will remain constant in an idle application

3. Garbage collection is not guaranteed to mark all the blocks is can as unused in one pass, which means that memory may never returned to its initial point

For a more details about the Flash garbage collector Grant Skinners has several writings about it in his AS Resource Management series: http://www.gskinner.com/blog/archives/2006/06/as3_resource_ma.html

So why do I care about garbage collection?

It doesn’t always collect what you want it to when you want it to, and when dealing especially with modules it can lead to some behaviors that look like memory leaks. I have personally found that if you want to get memory under control with modules you absolutely have to force garbage collection. For example consider a test application in which I am loading and unloading the same three modules over an over again into the current application domain. I know, what is an application domain? For now don’t worry about because it is complicated and something that I intend to explain later in another article. All you need to be aware of that this is the default module behavior if you use the Module Manager to load a module without using any parameters.

Without getting into the details of application domains and their specific effects on memory behavior, you can at least see that if you don’t force garbage collection memory just keeps going up with little breaks in between. In the case of the application that I am describing it will eventually cause my browser to run out of memory and lock up once the Flash Player gets upwards of 120 MB.

I have also found that the larger the modules and the more detailed the styles the more pronounced this problem. I believe that when I averaged out the increase in memory over time, it ended up being about 1.5 MB per module load. If you force garbage collection after each module unload it ends up being pretty much a flat line, granted though other techniques were incorporated in this application to help memory along to be discussed in future articles. The important thing to note is the positive effect forced garbage collection has by itself.

Force garbage collection?

Junior Java programmers love to ask the question about how to force garbage collection, to which the reply is always that you can only suggest it. The same is not true for Flex, as there are unsupported techniques that have various effects in the debug and non-debug Flash Player as well as with AIR.

Method 1: System.gc()

According to the Adobe documentation (http://www.gskinner.com/blog/archives/2006/06/as3_resource_ma.html) this forces garbage collection, but only in AIR and the debug version of the Flash Player. In practice though this method doesn’t seem to be very effective for reasons listed below.

Method 2: System.gc() twice in a row

According to Sean Christmann’s post about kick starting the garbage collector (http://www.craftymind.com/2008/04/09/kick-starting-the-garbage-collector-in-actionscript-3-with-air/), if you call System.gc() twice in a row it is far more effective. Unfortunately though this only works in the debug Flash Player and AIR.

Method 3: The local connection hack

According to Grant Skinner’s third installment of this AS Resource Management Series (http://www.gskinner.com/blog/archives/2006/08/as3_resource_ma_2.html), you can force garbage collection by calling two local connections in a row:

try {

   new LocalConnection().connect('foo');

   new LocalConnection().connect('foo');

} catch (e:*) {}

In testing this I find that it works in AIR, the debug Flash Player, and the non-debug Flash Player. It is also the my preferred way of forcing garbage collection, even though it it not supported.

When do I force garbage collection?

I have engaged in several debates regarding when to force garbage collection and why, and the most important thing you need to understand is that forcing garbage collection will only clear objects that are eligible for collection in the first place. This means that if you write a loop in your application to force garbage collection at 70 MB until it goes down to 50 MB you are probably going to loop infinitely. Forcing garbage collection is not the silver bullet to all of your memory problems.

When to force garbage collection really depends on what your application is doing. For example if your application just consists of several screens you might try forcing garbage collection in between screens. In other applications it may be more appropriate to force it every few minutes.

How do I make objects eligible for garbage collection?

Now that is the million dollar question, and unfortunately there is no one answer. The purpose of this series on Flex memory issues is designed to help one thing at a time. Binding, states, modules, effects, application domains, complex components, listeners, and more will have their memory issues explained.

Flex Memory Issue #2: Component effects cause leaks

2009-05-27T16:05:00.001-06:00

An effect is a change to a component that occurs over a brief period of time (http://www.adobe.com/devnet/flex/quickstart/adding_effects/). Flex provides a series of predefined effects which you can be played individually, simultaneously, and one after another as a group. They are useful for adding that “wow” factor to your application by giving it animation. There is a problem though with using these effects, which is that if you remove a component that has had an effect played on it that component will still reside in memory until another effect is played.

This is because all effects are controlled through the EffectManager, which keeps track of the last effect played through the “lastCreatedEffect” property. This may not seem like a big deal since it will just change when you play another effect, but is a big deal of you are limited on the amount of memory you can use and the component being held onto is large. It is also an annoyance if you are trying to track down leaks and components that shouldn’t be there because of this are floating around in memory.

The solution to this problem is pretty simple, which is to just call EffectManager.lastCreatedEffect = null when you are done with your effect. After figuring this out on my own I Googled around and was able to find that someone created a defect for it back in 2008.

Component effects via EffectManager cause memory leak (http://bugs.adobe.com/jira/browse/SDK-18061)

Steps to reproduce:
1. Create an MXML component with an effect, e.g.
<mx:HBox id="myBox" resizeEffect="{new Resize()}"/>
2. Use the component such that the effect gets played, e.g. set its size explicitly in this case
3. Remove the component.

Actual Results: the component is not garbage collected.

Expected Results: the component is garbage colloected.

Workaround (if any): Explicitly register an EFFECT_START listener on the effect to set EffectManager.lastCreatedEffect = null when the effect plays.

This leak occurs because lastCreatedEffect is a static variable and thus hangs on the the last played effect, which then hangs on to its target component, etc. I recommend removing this static field completely. If it is really needed, make it a weak reference instead.

Flex Memory Issue #1: The debug player always leaks

2009-05-26T19:27:00.001-06:00

In the 9 and 10 versions of the Flash Player there are both debug and non-debug versions of those players. If you are writing Flex application you most certainly have the debug version, which lets you output to trace as well as write those trace statements to a log on the local file system of you have the correct settings. There is a problem with the debug version of the Flash Player though, which is that it is full of memory leaks.

This is important if you are trying to get memory under control in a Flex application, because if you are just looking at the System.totalMemory in the debug player you are wasting your time. This problem with memory and the debug player is most evident if you are using modules, which you can see by loading modules over and over again and comparing System.totalMemory over time in the debug and non-version of the Flash Player.

Using the latest in computer modeling technology the result looks like the following:

This means that if you want to get a realistic picture of memory behavior using System.totalMemory, or by looking at the effect on the containing browser or AIR executable, you have to use the non-debug version of the Flash Player. The debug version of the Flash Player is still useful for looking at memory usage though, but you have to use the profiler.

The profiler memory usage panel displays the amount of memory used by object allocation, and will not correlate with System.totalMemory. How do I know this? Well, I took the time to add up all of the objects listed in Live Objects panel and it always adds up to the amount listed for Current Memory. Overtime System.totalMemory will further diverge from the displayed Current Memory, because the actual memory used will continue to grow because you are using the debug player.

I was never able to find a good explanation of the huge difference between Current Memory and the actual memory consumed by the application when running with the profiler versus the non-debug version of the Flash Player, but I have also found that the memory usage overtime of an application in the profiler according to Current Memory will roughly match the memory usage of that same application overtime run using the non-debug version of the Flash Player. It is a rough match because there is of course other stuff in memory than just object allocations.

The non-debug version of the Flash Player is good for locating a generic problem with memory at the level of “hey, this little application started at 30 MB and is now at 300 MB,” while the profiler is good for narrowing down which instances are leaking. When investigating memory in Flex applications you just have to be aware of how whether you are using the debug or non-debug version of the Flash Player effects your results.

Flex: event.target versus event.currentTarget

2009-05-26T18:04:00.001-06:00

The event system in Flex allows you catch an Event and its instance to get reference to the component that generated the event.

For example if you do make the following declaration in MXML:

<mx:Button click=”clicked(event)” />

When that button is clicked it calls in ActionScript the function “clicked(event:MouseEvent):void”

So how do you get reference to the button that generated the event in that function?

Well, according to the Adobe Event API (http://livedocs.adobe.com/flex/3/langref/flash/events/Event.html) you can use:

event.target - The event target. This property contains the target node. For example, if a user clicks an OK button, the target node is the display list node containing that button.
event.currentTarget - The object that is actively processing the Event object with an event listener. For example, if a user clicks an OK button, the current target could be the node containing that button or one of its ancestors that has registered an event listener for that event.

So what does that mean and which should I use?

You can’t tell from that description, but I can tell you about a problem I was having with a component displaying lots of buttons. So there was a component with lots of buttons on it, where when a button was pressed it would call a function that would determine which button was pressed and do all sorts of fancy stuff.

The callback function looked something like the following:

private function clicked(event:MouseEvent):void

{

var button:button = event.target as Button;

trace(button.label);

}

The problem with this was that if the user started clicking on buttons really fast a null pointer exception would occur, because the “button” instance was null. After debugging this I realized that the button was null because for a few cases when the button was clicked the event.target was a UITextField, what?

That is when a stumbled onto some more event documentation currently located at http://livedocs.adobe.com/flex/3/html/help.html?content=16_Event_handling_5.html:

The target property

The target property holds a reference to the object that is the target of the event. In some cases, this is straightforward, such as when a microphone becomes active, the target of the event object is the Microphone object. If the target is on the display list, however, the display list hierarchy must be taken into account. For example, if a user inputs a mouse click on a point that includes overlapping display list objects, Flash Player and AIR always choose the object that is farthest away from the Stage as the event target.

For complex SWF files, especially those in which buttons are routinely decorated with smaller child objects, the target property may not be used frequently because it will often point to a button's child object instead of the button. In these situations, the common practice is to add event listeners to the button and use the currentTarget property because it points to the button, whereas the target property may point to a child of the button.

The currentTarget property

The currentTarget property contains a reference to the object that is currently processing the event object. Although it may seem odd not to know which node is currently processing the event object that you are examining, keep in mind that you can add a listener function to any display object in that event object's event flow, and the listener function can be placed in any location. Moreover, the same listener function can be added to different display objects. As a project increases in size and complexity, the currentTarget property becomes more and more useful.

Pay attention to the following statement in the section about the target property:

“…common practice is to add event listeners to the button and use the currentTarget property because it points to the button, whereas the target property may point to a child of the button.”

So after reading this I changed my code to use event.currentTarget from event.target, and it always returned the instance of the button that was clicked.

I also found another article (http://mysticnomad.wordpress.com/2008/03/21/difference-between-eventtarget-and-eventcurrenttarget-properties-in-an-event-object/) that clearly laid out the distinction between these two properties, and made the conclusion that “…you should always use event.currentTarget property instead of event.target.”

I agree.

Command Module Framework

2009-05-10T15:29:00.001-06:00

What is it?

The Command Module Framework is a series of classes for working with modules in Flex, which extends the default module behavior to compensate for numerous known problems as well as to add needed functionality.

In particular the following is a list of some of the more important functional differences:

The ability to queue events on the module loader which get dispatched to the module implementation once it is loaded and ready
Modules when unloaded go through a series of magic tricks to help them actually unload. Some day I will get around to explaining what a lot of these tricks are and why they are needed.
The ability to configure a manager to handle loading and unloading modules in view stacks
The built in ability to share data between all modules
The built in ability to optionally display module loading progress
Makes it easier to communicate between modules

What is it intended to be used for?

This library is intended to be used for Flex applications that need to load and unload modules, with a lot of the common code used for doing this already written.

Modules are used so that the user of your application only has to download the parts they need when they need it. For example if an application is a series of 100 different forms, it makes sense that the user should only have to download the forms they intend to use. The is a big deal in web applications because is means the difference (depending on your connection speed) between spending 15 minutes downloading a large application versus spending 5 seconds downloading an application framework and 3 seconds for each additional module.

How does it work?

There are two general ways to use it:

Using the Command Module Loader, which is similar to the Module Loader
Using the Command Module Manager, which is similar to the Module Manager

Using the Command Module Loader (Example #1)

The Command Module Loader is very similar to the Module Loader, and can be used as in the same way in a basic example:

Application.mxml

The code for Module A is the following:

ModuleA.mxml

<?xml version="1.0" encoding="utf-8"?>
<commandmodulefr:CommandModule
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*"
    xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Panel title="Module A" width="100%" height="100%">
        <mx:Button label="I am a button" />
    </mx:Panel>
</commandmodulefr:CommandModule>

The result is an application that loads the module at the given URL, assuming it inherits from the CommandModule class, which in the case of this module results in a panel with a button in it:

Sending events to modules (Example #2)

Let’s say that for some reason you want to send an event to your module, but being a smart Flex developer you are aware of the following:

I can’t reference an instance of my custom module class in my main application because that would cause that class to get compiled in with my main application, which would defeat the purpose of modules
If I dispatch an event directly to the loader using dispatchEvent, my module could miss that event if it is not already loaded

For these reasons the CommandModuleLoader class has a queueEvent function which ensures that the module always gets the event no matter when it is dispatched.

For this example there is a button in the main application that when pressed sends some custom event to the module:

Application.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application
    xmlns:mx="http://www.adobe.com/2006/mxml"
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*">
    <mx:Script>
        <![CDATA[
            private function sendSomeEvent():void
            {
                var event:Event = new Event("someCustomEvent");
                moduleLoader.queueEvent(event);
            }
        ]]>
    </mx:Script>
    <mx:Button label="Send Some Custom Event" click="sendSomeEvent()" />
    <commandmodulefr:CommandModuleLoader id="moduleLoader" url="valentino/modules/ModuleA.swf" />
</mx:Application>

In the module there is a listener created on initialization for the custom event, which when the event is dispatched displays a popup window:

ModuleA.mxml

<?xml version="1.0" encoding="utf-8"?>
<commandmodulefr:CommandModule
    initialize="onInit()"
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*"
    xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Script>
        <![CDATA[
            import mx.controls.Alert;
            private function onInit():void
            {
                this.addEventListener("someCustomEvent", someHandler);
            }
            private function someHandler(event:Event):void
            {
                Alert.show("Some Custom Event!");
            }
        ]]>
    </mx:Script>
    <mx:Panel title="Module A" width="100%" height="100%">
        <mx:Button label="I am a button" />
    </mx:Panel>
</commandmodulefr:CommandModule>

Receiving events from modules (Example #3)

In order to receive an event from a module you must attach some listener on the actual module, and being a smart Flex developer you are aware of the following:

I can’t reference an instance of my custom module class in my main application because that would cause that class to get compiled in with my main application, which would defeat the purpose of modules
Being able to reference an event function using an attribute in the MXML of the loader would require the loader to already have the desired event metadata declarations

For these reasons the base implementation of the loader has an event for when the module is loaded, which can be caught in order to obtain a reference to the CommandModule class. This instance can then have a listener for anything you want added to it.

Application.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application
    xmlns:mx="http://www.adobe.com/2006/mxml"
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*">
    <mx:Script>
        <![CDATA[
            import mx.controls.Alert;
            import net.sourceforge.commandmodulefr.CommandModule;
            import net.sourceforge.commandmodulefr.event.CommandModuleEvent;
            private function moduleLoaded(event:CommandModuleEvent):void
            {
                var module:CommandModule = event.module;
                module.addEventListener("someCustomEvent", someHandler);
            }
            private function someHandler(event:Event):void
            {
                Alert.show("Some event from a module!");
            }
        ]]>
    </mx:Script>
    <commandmodulefr:CommandModuleLoader
        moduleLoaded="moduleLoaded(event)"
        url="valentino/modules/ModuleA.swf" />
</mx:Application>

When the button is pressed in the module, it dispatches the expected event which is then picked up in the main application.

ModuleA.mxml

<?xml version="1.0" encoding="utf-8"?>
<commandmodulefr:CommandModule
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*"
    xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Script>
        <![CDATA[
            private function sendEvent():void
            {
                var event:Event = new Event("someCustomEvent");
                this.dispatchEvent(event);
            }
        ]]>
    </mx:Script>
    <mx:Panel title="Module A" width="100%" height="100%">
        <mx:Button label="I am a button" click="sendEvent()" />
    </mx:Panel>
</commandmodulefr:CommandModule>

The result is a window that is displayed when the button within the module is pressed.

Sharing data between modules and the main application (Example #4)

Besides just using a singleton, the Command Module Loader accepts an Application Context object that is passed to all of the modules that are loaded. This class contains a data attribute which is an object that can be populated with anything, or you can just extend this class and defined a shared structure.

In this example the main application declares the Application Context, and sets the data so that “someVar” has a value.

Application.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application initialize="onInit()"
    xmlns:mx="http://www.adobe.com/2006/mxml"
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*">
    <mx:Script>
        <![CDATA[
            import net.sourceforge.commandmodulefr.data.ApplicationContext;
            [Bindable]
            private var _context:ApplicationContext = new ApplicationContext();
            private function onInit():void
            {
                _context.data.someVar = "This is some text";
            }
        ]]>
    </mx:Script>
    <commandmodulefr:CommandModuleLoader
        context="{_context}"
        url="valentino/modules/ModuleA.swf" />
</mx:Application>

The module then uses that value from the Application Context in order to get the text that it displays on the button.

ModuleA.mxml

<?xml version="1.0" encoding="utf-8"?>
<commandmodulefr:CommandModule
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*"
    xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Panel title="Module A" width="100%" height="100%">
        <mx:Button label="{context.data.someVar}" />
    </mx:Panel>
</commandmodulefr:CommandModule>

The result is the following:

Using the Command Module Manager (Example #5)

The Command Module Manager is far more generic, and is actually intended to be used as part of some other framework in an application. The module manager class is useful for when you do not want to have to declare every module that you intended to use. An example usage would be in an application that has a list of the URL’s of all of the module that it can load in some external XML file, where various events in that application would result in modules from that XML file being loaded.

The two most notable things about the Command Module Loader class are the following:

It takes a view stack as a parameter and loads and unloads module into and out of that view stack.
It cleans up modules so that they can actually be unloaded (somewhat, but this is for a later lengthy discussion)

Now consider an application that just has a series of buttons for loading and unloading modules, and a general area that the modular content is displayed in using a view stack. Though this usage seems really simple, it is a quite common template for an application.

Application.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application
    xmlns:mx="http://www.adobe.com/2006/mxml"
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*">
    <commandmodulefr:CommandModuleManager id="manager" viewStack="{view}" />
    <mx:Button label="Load Module A"
        click="manager.requestLoadModule('valentino/modules/ModuleA.swf')" />
    <mx:Button label="Unload Modules" click="manager.unloadAll()" />
    <mx:ViewStack id="view" />
</mx:Application>

The idea behind the module manager is to allow an application an easy way to change out dynamic content which comes from modules. The generic nature of the Command Module Manager then allows the individual application to decide about the details of how module communication occurs.

ModuleA.mxml

<?xml version="1.0" encoding="utf-8"?>
<commandmodulefr:CommandModule
    xmlns:commandmodulefr="net.sourceforge.commandmodulefr.*"
    xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:Panel title="Module A" width="100%" height="100%">
        <mx:Button label="I am a button" />
    </mx:Panel>
</commandmodulefr:CommandModule>

The result is a simple application that lets you load and unload modules using buttons from the main application.

In Practice

How you would go about using this library really depends on what you are doing, but for most implementations the Command Module Manager is probably going to be the most useful. This is because for any application of a medium to large size and for many reasons including maintainability and user experience, you are going to want to divide the application into modules. If you are using a considerable number of modules, the application is a lot easier to modify if the usage of those modules are externally defined.

For example if you have an application that has 100 modules and you wanted to use a module loader for each one, you would have to have 100 module loader declarations in your application. Every time someone added a module they would also have to add a module loader declaration, which would become very cumbersome. You could of course deal with the module loaders programmatically but you would essentially be recreating the equivalent of a module manager.

A technique which I have used on various projects that seems to be pretty straight forward is to have some external file, almost always XML, which specifies the module locations as well as other information. This file at its most basic level is just a way to declare module names and locations outside of your code, giving more flexibility at runtime.

For example if I had the following file:

<Module name=”ModuleA” url"=”valentino/modules/ModuleA.swf” />

<Module name=”ModuleB” url"=”valentino/modules/ModuleB.swf” />

<Module name=”ModuleC” url"=”valentino/modules/ModuleC.swf” />

</Modules>

I would then write some type of manager (referred to as a State Process Manager) class in my application to listen for particular events, and then based on those events look up a module’s URL in the external XML file and then pass them to the Command Module Loader to do the loading.

The resulting architecture looks something like the following:

State Process Manager - The objective of the state process manager is to control the event flow of the application, requiring reference to the one or more command module managers, the external module definitions, and the modules.

Command Module Manager - The command module manager is just an easy way to load an unload modules in and out of a view stack using URL’s.

Module Definitions – A list of the URL’s that contain the modules to load, which can also include any other information regarding module runtime configuration.

Modules – Portions of the UI to be loaded and unloaded.

Link to the Open Source Project

https://sourceforge.net/projects/commandmodulefr/

FlexB – XML mapping for Flex/ActionScript classes and web services

2008-12-21T22:39:00.001-06:00

Flex has built it support for working with XML as well as built in support for mapping XML to ActionScript classes and back to XML again if you have an XSD file. What about mapping to and from XML if you don’t have an XSD file, and what about this XML web service stuff? FlexB is a Flex/ActionScript library for more easily working with XML and web services. It consists of a combination of classes that I originally made for my own personal use while teaching myself Flex a little over 2 years ago, which I have recently decided to renovate and make public.

Where can I get the code, library, and/or participate?

http://flexb.sourceforge.net

Why don’t you just use an XSD file and the built in XML mappings?

Because I am lazy and don’t want to use an XSD file. There are also equivalents to this idea in other languages such as with XStream and Java.

Why map XML to ActionScript classes with all of the direct component XML support?

Components such as the DataGrid allow you to directly reference XML in order to do useful things with your data. The following example shows how to populate a DataGrid using XML:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:XML xmlns="" id="CompoundFoo">
        <CompoundFoo my_name="wii">
            <SimpleFoo><name>Moo Cow</name><foo>Bar</foo></SimpleFoo>
            <SimpleFoo><name>Meow Cat</name><foo>BarBar</foo></SimpleFoo>
        </CompoundFoo>
    </mx:XML>
    <mx:DataGrid dataProvider="{CompoundFoo.SimpleFoo}">
        <mx:columns>
            <mx:DataGridColumn headerText="Name" dataField="name" />
            <mx:DataGridColumn headerText="Foo" dataField="foo" />
        </mx:columns>
    </mx:DataGrid>
</mx:Application>

Using XML directly in a DataGrid is one of the fastest ways possible to display data in a Flex application, but what about maintainability? The XML in the DataGrid is a popular Flex example, but it is unrealistic. In a real world application that XML or data will most likely come from some external source, an external source which is subject to change. It is also likely that the data will be used in the application in more than once place, and its usage is likely to involve other customizations. It is also common for data to need to be converted to other types, such as with a String to a Date. Mapping XML to ActionScript classes and then using those classes makes change easier, especially if those mapped classes are used in several places throughout the application.

How is mapping XML to ActionScript better for change then mapping XML directly to several components like the DataGrid?

If you were to change any part of the XML from the first example, you would then have to change all of these references to it; the dataProvider, dataFields, and any other references. Use that XML in several places and you have several changes to make. These direct references to the XML are mapping, which can be spread out throughout components. If you use an ActionScript class to do the mappings, then all of your mappings are in the same place so it is easier to find and change. Since references are then to ActionScript classes you also gain the advantage of those references being type and name safe.

For example a reference to SimpleFoo.foo as XML would always compile whether it existed or not. If there were a class called SimpleFoo then access to the property foo would cause a compile error if it didn’t exist.

The following example shows the same DataGrid and XML, but this time the DataGrid is populated using an ActionScript class:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application
    xmlns:valentino="valentino.*"
    xmlns:mx="http://www.adobe.com/2006/mxml">
    <mx:XML xmlns="" id="xml">
        <CompoundFoo my_name="wii">
            <SimpleFoo><name>Moo Cow</name><foo>Bar</foo></SimpleFoo>
            <SimpleFoo><name>Meow Cat</name><foo>BarBar</foo></SimpleFoo>
        </CompoundFoo>
    </mx:XML>
    <valentino:CompoundFoo id="compoundFoo" xml="{xml}" />
    <mx:DataGrid dataProvider="{compoundFoo.foos}">
        <mx:columns>
            <mx:DataGridColumn headerText="Name" dataField="{valentino.SimpleFoo.NAME}" />
            <mx:DataGridColumn headerText="Foo" dataField="{valentino.SimpleFoo.FOO}" />
        </mx:columns>
    </mx:DataGrid>
</mx:Application>

This example uses the ActionScript classes CompoundFoo and SimpleFoo to contain all the instructions for XML Mapping. In the event that the XML were to change the only places in the application that would have to change would be those mappings. All current references to that data throughout the application could remain the same.

The difference is where you do the mapping. FlexB places the mappings in the ActionScript classes so that those are the only places that have change.

How does the XML mapping work?

My preference would be to eventually have a factory class look at annotations of plain old ActionScript classes and do the mappings, but for right now I am providing a class to extend that gives this mapping functionality. Using inheritance also provides the ability to easily override all of the mapping behavior.

The following ActionScript class defines the node, attribute, and class mappings from a “CompoundFoo” XML document:

package valentino
{
import net.sourceforge.flexb.core.IMap;
import net.sourceforge.flexb.mapper.XmlMappedObject;

    [Bindable]
    public class CompoundFoo extends XmlMappedObject
    {
        public var myName:String;
        [ArrayElementType("valentino.SimpleFoo")]
        public var foos:Array = new Array();
        override protected function mapNodeToFunction(nodeToFunctionMap:IMap):void
        {
            nodeToFunctionMap.put("SimpleFoo", "foos");
        }
        override protected function mapAttributeNodeToFunction(attributeNodeToFunctionMap:IMap):void
        {
            attributeNodeToFunctionMap.put("my_name", "myName");
        }
        override protected function mapNodeToClass(nodeToClassMap:IMap):void
        {
            nodeToClassMap.put("SimpleFoo", SimpleFoo);
            nodeToClassMap.put("CompoundFoo", CompoundFoo);
        }
    }
}

The node to function mappings specify the name of each node and the set/get functions of the class to map to. For example when the node “SimpleFoo” is encountered during mapping its value is associated with the class property of “"foos.”

The node to attribute mappings specify the name of each attribute and the set/get functions of the class to map to. For example when the attribute “my_name” is encountered during the mapping its value is associated with the class property of “myName.”

The node to class mappings specify the name of each node and the associated ActionScript class. For example when the node “SimpleFoo” is encountered during the mapping its value is mapped as the “SimpleFoo” ActionScript class.

Behavior such as needed for the conversion of data can also be handled by using set/get functions instead of providing publically accessible class variables.

“SimpleFoo” is also a reference to another ActionScript class:

package valentino
{
import net.sourceforge.flexb.core.IMap;
import net.sourceforge.flexb.mapper.XmlMappedObject;

    public class SimpleFoo extends XmlMappedObject
    {
        public var name:String;
        public var foo:String;
        public static const FOO:String = "foo";
        public static const NAME:String = "name";

        override protected function mapNodeToFunction(nodeToFunctionMap:IMap):void
        {
            nodeToFunctionMap.put(FOO, "foo");
            nodeToFunctionMap.put(NAME, "name");
        }
        override protected function mapNodeToClass(nodeToClassMap:IMap):void
        {
            nodeToClassMap.put("SimpleFoo", SimpleFoo);
        }
    }
}

What about web service mapping?

It is actually an HTTPService mapping, and it is just an easy way to get XML data from an external source and map it to an ActionScript class. In FlexB it is called XmlService.

There are 3 ways to use it:

1. Just like an HTTPService where you handle the ResultEvent and call the mapping functions.

<service:XmlService
        id="xmlServiceResultID"
        result="xmlServiceResult(event)"
        url="../testdata/CompoundFoo.xml" />

private function xmlServiceResult(event:ResultEvent):void
{

var compoundFoo:CompoundFoo = new CompoundFoo();
compoundFoo.map(XML(event.result));
}

2. Specify to handle an XmlResultEvent which does the mapping for you.

<service:XmlService
        id="xmlResultService"
        resultClass="{CompoundFoo}"
        xmlResult="onXmlResult(event)"
        url="../testdata/CompoundFoo.xml" />

private function onXmlResult(event:XmlResultEvent):void
{
var compoundFoo:CompoundFoo = event.xmlMappedObject as CompoundFoo;
}

3. Specify a function to call with the mapped instance of the object.

<service:XmlService
        id="xmlFunctionService"
        resultClass="{CompoundFoo}"
        resultFunction="{xmlServiceXmlResult}"
        url="../testdata/CompoundFoo.xml" />

public function xmlServiceXmlResult(compoundFoo:CompoundFoo):void
{

}

In Response to the Easiest Java XML Binding. What About XStream?

2008-07-06T13:05:00.001-06:00

Originally I asked the following question in this posting:

I have an XML document and I want to use that document to populate a corresponding set of Java objects. This is a commonly encountered scenario when working with Java, so what is the easiest method for Java XML Binding that requires the least amount of code?

What do I want? The 5 requirements of Java XML laziness.

I then went on to explain past methods of XML binding in Java, and what it was that I was exactly looking for. To summarize that endeavor, I was unable to locate something that fit my exact needs:

I don't want to have to manually iterate through XML documents and manually map values.
I don't want to have to use XSD files.
I don't want to have to use libraries for binding compilers to use external binding definitions to map to Java classes.
I don't want to have to modify existing XML documents so that I can use them as beans.
I want to be able to have class and class field names that are different then their corresponding XML node names.

What is available?

I was aware that something had to exist that did this; I just couldn't find it. I searched the usual places such as sourcefoge.net, google, java.sun.com and so on, but everywhere I went I was bombarded with results that didn't fit my exact needs. After posting regarding this conundrum and my solution to it, which was to contribute to the problem of too many Java XML bindings by creating my own, several recommendations were made to me. The following is a list of Java XML bindings that have used and that were recommended to me:

JAXB - Requires Java Web Services Developer Pack v1.1 or later and the use of XSD's to bind the corresponding XML to Java interfaces (http://java.sun.com/developer/technicalArticles/WebServices/jaxb/).
JiBX - Open source project that uses binding definition documents to describe how XML is mapped to Java objects at binding runtime, where enhanced class files generated by the binding compiler build objects from an XML input document and can be outputted as XML documents (http://jibx.sourceforge.net/).
SAX/DOM/JAXP/Xerces-J - These are all types of Java XML parsers that require the manual handling of XML documents, meaning that you have to [in general] manually map the values from an XML document to the corresponding Java object (http://www.cafeconleche.org/books/xmljava/chapters/ch05.html).
Spring Beans - Uses the Inversion of Control container from the Spring Framework in order to bind values from special "bean" xml documents to Java classes. This is not useful for reading a specific XML document, but it is the binding that is of interest ; the automatic mapping of XML value to Java object field. (http://jvalentino.blogspot.com/2007/10/introduction-to-spring-framework.html).
EJB Beans - Enterprise Java Beans are a subject unto themselves, far to broad for discussion here. What is important though as with the inversion of control container from the Spring Framework, is the ability to bind values from XML documents to Java object fields (http://java.sun.com/javaee/5/docs/tutorial/doc/bnblr.html).
Castor - Uses marshalling and unmarshalling to go from XML to POJO and POJO to XML. The documentation is unfinished, but from what I can gather in order to go from XML to POJO and POJO to XML the POJO needs to implement serializable, and if the XML node names do not match class field names a binding file has to be used (http://www.castor.org/xml-framework.html).
Codehaus XFire - Another method for going from XML to POJO and POJO to XML, and requires the use of XSD files to do so (http://xfire.codehaus.org/Aegis+Binding).
XStream - Another method for going from XML to POJO and POJO to XML, but instead of having to use XSD files and other forms of external mappings it allows the use of annotations for aliasing (http://xstream.codehaus.org/).

What about XStream?

As it turns out XStream was exactly was I was looking for; it meets my 5 requirements of Java XML laziness. So how does it work?

Consider an RSS 2.0 document and what it represents:

<?xml version="1.0"?>
<rss version="2.0">
  <channel>
    <title>Lift Off News</title>
    <link>http://liftoff.msfc.nasa.gov/</link>
    <description>Liftoff to Space Exploration.</description>
    <language>en-us</language>
    <pubDate>Tue, 10 Jun 2003 04:00:00 GMT</pubDate>
    <lastBuildDate>Tue, 10 Jun 2003 09:41:01 GMT</lastBuildDate>
    <docs>http://blogs.law.harvard.edu/tech/rss</docs>
    <generator>Weblog Editor 2.0</generator>
    <managingEditor>editor@example.com</managingEditor>
    <webMaster>webmaster@example.com</webMaster>
    <ttl>5</ttl>
 
    <item>
      <title>Star City</title>
      <link>http://liftoff.msfc.nasa.gov/news/2003/news-starcity.asp</link>
      <description>How do Americans get ready to work with Russians aboard the
        International Space Station? They take a crash course in culture, language
        and protocol at Russia's Star City.</description>
      <pubDate>Tue, 03 Jun 2003 09:39:21 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/06/03.html#item573</guid>
    </item>
 
    <item>
      <title>Space Exploration</title>
      <link>http://liftoff.msfc.nasa.gov/</link>
      <description>Sky watchers in Europe, Asia, and parts of Alaska and Canada
        will experience a partial eclipse of the Sun on Saturday, May 31st.</description>
      <pubDate>Fri, 30 May 2003 11:06:42 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/05/30.html#item572</guid>
    </item>
 
    <item>
      <title>The Engine That Does More</title>
      <link>http://liftoff.msfc.nasa.gov/news/2003/news-VASIMR.asp</link>
      <description>Before man travels to Mars, NASA hopes to design new engines
        that will let us fly through the Solar System more quickly.  The proposed
        VASIMR engine would do that.</description>
      <pubDate>Tue, 27 May 2003 08:37:32 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/05/27.html#item571</guid>
    </item>
 
    <item>
      <title>Astronauts' Dirty Laundry</title>
      <link>http://liftoff.msfc.nasa.gov/news/2003/news-laundry.asp</link>
      <description>Compared to earlier spacecraft, the International Space
        Station has many luxuries, but laundry facilities are not one of them.
        Instead, astronauts have other options.</description>
      <pubDate>Tue, 20 May 2003 08:56:02 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/05/20.html#item570</guid>
    </item>
  </channel>
</rss>

Document from http://en.wikipedia.org/wiki/RSS_(file_format)

An "rss" node has a "channel" node, a "channel" node has "item" nodes, and every node has their own properties and attributes. These relationships can then be represented in terms of objects using a class diagram (ignoring methods for getters and setters):

From this class diagram it is then possible to generate the corresponding Java classes, assuming that the class fields have the same names as their corresponding XML nodes. This can't always work though since class fields can be represented in XML as nodes or attributes of nodes, so it would be necessary to designate this in Java. You may also not want to have the Java class field names the same as the XML node names, so there would need to be a way to designate this as well. This is why there will always need to be some form of mapping, which is why XStream provides annotations and also methods for their service to specify aliasing at runtime.

Bad use of annotations?

With my previous posting where I used annotations in the same manner as XStream, there was discussion regarding whether this use of annotations was necessary. The answer is absolutely yes for two reasons:

For when XML node names are not the same as class and class field names.
In order to translate the representation of a class field as an XML node or XML node attribute.

Consider the following class and the following XML document:

public class Example {
 public String myValue;
 public String version;
}

<example ver="1.0">
 <my_value>value</my_value>
</example>

How would one map the XML to the class when the class name and the field names are different then the XML node names? While some XML documents may be able to have their class field names interfered based on similarities, types, and positions this would not work for all cases. This is one reason why annotations or other mechanisms are need to map between class fields and XML nodes.

Also consider when converting the class to XML, how would one determine whether a class field should be represented as an XML node or a node attribute? The following XML documents all could be used to represent the data in the Example class:

<example>
 <ver>1.0</ver>
 <my_value>value</my_value>
</example>

<example ver="1.0" my_value="value" />

<example my_value="value">
 <ver>1.0</ver>
</example>

How does XStream do what you need?

XStream allows the use of annotations or object aliasing methods to specify mappings and attributes. For example here is my previous RSS 2.0 POJP example marked up for XStream using annotations:

@XStreamAlias("rss")
public class Rss2 {
 @XStreamImplicit
 private List<Channel> channel;
 @XStreamAsAttribute
 private String version;
 
 //getters and setters go here...
}

@XStreamAlias("channel")
public class Channel {
 private String title;
 private String link;
 private String description;
 private String language;
 private String pubDate;
 private String lastBuildDate;
 private String docs;
 private String generator;
 private String managingEditor;
 private String webMaster;
 private int ttl;
 @XStreamImplicit
 private List<Item> item;
 
 //getters and setters go here...
}

@XStreamAlias("item")
public class Item {
 private String title;
 private String link;
 private String description;
 private String pubDate;
 private String guid;
 
 //getters and setters go here...
}

The following code can be used to populate an RSS object using XML:

XStream xstream = new XStream(new DomDriver());
xstream.processAnnotations(Rss2.class);

//Create an RSS object and populate it using an XML file
Rss2 rssA = new Rss2();
xstream.fromXML(new FileReader(new File("test/Rss2Test.xml")), rssA);

The following code can then be used to convert a POJO to XML:

String xml = xstream.toXML(rssA);

Why did you reinvent the wheel with your RE: JAXB project?

As I have said before I created a factory for binding XML to POJOs and back because I could not find anything at the time that suited my exact needs. To me it is really not that big of a deal, because it only took a single afternoon and only consisted of several hundred lines of code.

The Easiest Java XML Binding

2008-06-30T21:28:00.001-06:00

I have an XML document and I want to use that document to populate a corresponding set of Java objects. This is a commonly encountered scenario when working with Java, so what is the easiest method for Java XML Binding that requires the least amount of code? You can't answer that question without first defining what "easy" is in relation to the bindings between Java and XML. To me "easy" is synonymous with "simple," so I am equating easiness to complexity. Complexity is of interest because things that are complex take longer to implement and are generally harder to maintain, based on my experiences with software in a variety of languages and platforms. Less code written means less code to maintain, and less code to write means less time is required to write that code.

So what are the methods for binding Java and XML?

JAXB - Requires Java Web Services Developer Pack v1.1 or later and the use of XSD's to bind the corresponding XML to Java interfaces (http://java.sun.com/developer/technicalArticles/WebServices/jaxb/).
JiBX - Open source project that uses binding definition documents to describe how XML is mapped to Java objects at binding runtime, where enhanced class files generated by the binding compiler build objects from an XML input document and can be outputted as XML documents (http://jibx.sourceforge.net/).
SAX/DOM/JAXP/Xerces-J - These are all types of Java XML parsers that require the manual handling of XML documents, meaning that you have to [in general] manually map the values from an XML document to the corresponding Java object (http://www.cafeconleche.org/books/xmljava/chapters/ch05.html).
Spring Beans - Uses the Inversion of Control container from the Spring Framework in order to bind values from special "bean" xml documents to Java classes. This is not useful for reading a specific XML document, but it is the binding that is of interest ; the automatic mapping of XML value to Java object field. (http://jvalentino.blogspot.com/2007/10/introduction-to-spring-framework.html).
EJB Beans - Enterprise Java Beans are a subject unto themselves, far to broad for discussion here. What is important though as with the inversion of control container from the Spring Framework, is the ability to bind values from XML documents to Java object fields (http://java.sun.com/javaee/5/docs/tutorial/doc/bnblr.html).

I have used all of these methods in depth before, but this isn't exactly what I want though:

I don't want to have to manually iterate through XML documents and manually map values.
I don't want to have to use XSD files.
I don't want to have to use libraries for binding compilers to use binding definitions to map to Java classes.
I don't want to have to modify existing XML documents so that I can use them as beans.

My usage of Adobe Flex (http://jvalentino.blogspot.com/2008/02/flex-tutorial-part-ii-language-concepts.html) over the last couple of years has spoiled my expectations for language and XML interaction; I want more with less.

I know exactly what I want: I want to have a series of Java objects that are representations of nodes within an XML document, and pass the XML document to the Java object representing the root XML node and have all of the corresponding Java objects populate using that XML. I then want to be able to go from Java back to XML, and I don't want to have to do anything other then say "XML, go to Java" and then "Java, go to XML."

Why should there need to be complex mappings and external definitions when Java and XML are being used to representing the same thing?

Consider an RSS 2.0 document and what it represents:

<?xml version="1.0"?>
<rss version="2.0">
  <channel>
    <title>Lift Off News</title>
    <link>http://liftoff.msfc.nasa.gov/</link>
    <description>Liftoff to Space Exploration.</description>
    <language>en-us</language>
    <pubDate>Tue, 10 Jun 2003 04:00:00 GMT</pubDate>
    <lastBuildDate>Tue, 10 Jun 2003 09:41:01 GMT</lastBuildDate>
    <docs>http://blogs.law.harvard.edu/tech/rss</docs>
    <generator>Weblog Editor 2.0</generator>
    <managingEditor>editor@example.com</managingEditor>
    <webMaster>webmaster@example.com</webMaster>
    <ttl>5</ttl>
 
    <item>
      <title>Star City</title>
      <link>http://liftoff.msfc.nasa.gov/news/2003/news-starcity.asp</link>
      <description>How do Americans get ready to work with Russians aboard the
        International Space Station? They take a crash course in culture, language
        and protocol at Russia's Star City.</description>
      <pubDate>Tue, 03 Jun 2003 09:39:21 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/06/03.html#item573</guid>
    </item>
 
    <item>
      <title>Space Exploration</title>
      <link>http://liftoff.msfc.nasa.gov/</link>
      <description>Sky watchers in Europe, Asia, and parts of Alaska and Canada
        will experience a partial eclipse of the Sun on Saturday, May 31st.</description>
      <pubDate>Fri, 30 May 2003 11:06:42 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/05/30.html#item572</guid>
    </item>
 
    <item>
      <title>The Engine That Does More</title>
      <link>http://liftoff.msfc.nasa.gov/news/2003/news-VASIMR.asp</link>
      <description>Before man travels to Mars, NASA hopes to design new engines
        that will let us fly through the Solar System more quickly.  The proposed
        VASIMR engine would do that.</description>
      <pubDate>Tue, 27 May 2003 08:37:32 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/05/27.html#item571</guid>
    </item>
 
    <item>
      <title>Astronauts' Dirty Laundry</title>
      <link>http://liftoff.msfc.nasa.gov/news/2003/news-laundry.asp</link>
      <description>Compared to earlier spacecraft, the International Space
        Station has many luxuries, but laundry facilities are not one of them.
        Instead, astronauts have other options.</description>
      <pubDate>Tue, 20 May 2003 08:56:02 GMT</pubDate>
      <guid>http://liftoff.msfc.nasa.gov/2003/05/20.html#item570</guid>
    </item>
  </channel>
</rss>

Document from http://en.wikipedia.org/wiki/RSS_(file_format)

From this class diagram it is then possible to generate the corresponding Java classes, assuming that the class fields have the same names as their corresponding XML nodes. This can't always work though since class fields can be represented in XML as properties or attributes, so it would be necessary to designate this in Java. You may also not want to have the Java class field names the same as the XML node names, so there would need to be a way to designate this as well. One easy way to designate specific things about classes, fields, and methods is to use annotations, so consider the following 3 annotations:

@ClassXmlNodeName - used on a Java class to specify its corresponding node name in an XML document. This is required in order to designate a class as being bound to an XML document.
@XmlAttributeName - used on a Java field to specify its corresponding attribute name in an XML document. This is required in order to specify that a particular class field should be represented in XML as an attribute instead of a property.
@XmlNodeName - used on a Java field to specify its corresponding node name in an XML document. This is only required when a Java class field name is different than its corresponding XML node name.

@ClassXmlNodeName("rss")
public class Rss {
 private List<Channel> channels;
 @XmlAttributeName("version")
 private String version;
 
 //getters and settings go here...
}

@ClassXmlNodeName("channel")
public class Channel {
 private String title;
 private String link;
 private String description;
 private String language;
 private String pubDate;
 private String lastBuildDate;
 private String docs;
 private String generator;
 private String managingEditor;
 private String webMaster;
 private int ttl;
 private List<Item> items;

 //getters and settings go here...
}

@ClassXmlNodeName("item")
public class Item {
 private String title;
 private String link;
 private String description;
     private String pubDate;
     private String guid;
    
     //getters and settings go here... 
}

With the exception of the annotations, these Java classes are just like any other data transfer objects (DTOs) that would be used to represent a RSS XML document. These annotations act as the mappings between Java and XML when names and types do not provide enough information. With this information, the class representing the XML document root node, and the XML document it is possible to recursively map XML nodes to Java class fields and Java class fields back to XML nodes using Reflection. So using Reflection I wrote a library that can take any Plain Old Java Object (POJO) and an XML document, and use that XML document to populate that object and all of its child objects using one line of code. The same can then be done to convert a Java object back to its XML representation using a single line of code. All of this assuming that the Java classes correspond to the XML document.

Using this "XML Binder" library the following code takes the the given instance of an Rss object and populates it and its channels and items using the given XML document:

Rss rss = new Rss();
XmlBinderFactory.newInstance().bind(rss, new File("Rss2Test.xml"));

The following code can then be used to take that same Rss object once changes have been made to it and converts it back to XML:

String xml = XmlBinderFactory.newInstance().toXML(rss);

I assert that this can work with most XML documents that have appropriate corresponding Java classes, but I have only tested it out so far with RSS 1.0, RSS 2.0, and a general test XML document in no particular format. The question now is what do I do with this library; anyone interested?

Since there seems to be some interest I have started a sourceforge.net project called "Really Easy Java XML Binding" or "RE:JAXB" since this is sort of a reply to JAXB. Here is the link

I have included POJOs for RSS 1.0 and RSS 2.0 along with unit tests to verify that the mappings from XML and to XML are working correctly. What I am looking for are ways to improve the efficiency of the to and from binding code, as well as POJOs for other common XML formats.

The Flexsite

2008-05-17T16:54:00.014-06:00

Social networking websites such as MySpace and Facebook have many common themes regarding the reasons for their popularity. One common theme in particular is that each website gives the average user the ability to have a persistent, customizable, and maintainable online presence. These websites though typically are intended for individual users, meaning that there is not anything equivalent for businesses. If a business wants a website they have to either already have someone or hire someone to build and maintain the website. The more visually appealing and interactive that a websites needs to be the higher the cost and the more difficult it is to maintain. Flexsite solves this problem by providing a highly configurable, visually appealing, inexpensive, and easily maintainable website that operates like familiar social networking websites for small businesses with all of the setup included.

Flexsite uses numerous web technologies in order to provide services and interact with existing services:

• HTML
• XML
• Flash
• DTD
• Flex
• RSS
• WordPress
• PayPal
• PHP
• MySQL
• JavaScript
• CSS

The Flexsite application itself is divided into systems based on function, which incorporate one or more of the previously listed technologies and/or third party services. The visuals for Flexsite are done as an Adobe Flex application, which communicates with other services specified by the XML layout vocabulary. All communication with the Adobe Flex application is done through XML as well. Of all of the systems only the XML layout vocabulary is required, everything else is optional and Flexsite can run without them. The following is a diagram of all the involved systems:

Figure 1: Systems

The XML layout vocabulary is just an XML document used to specify website sizing, sections, menu items, content, Flash skins, and web services. There is also a corresponding DTD which is used to validate the layout XML. For web services that support PHP there are Flexsite Services that can be used to manipulate the XML using an Adobe Flex graphical user interface.

Figure 2: Layout Manager

Flexsite also incorporates the use of Flash Skins for the look and feel of the website. Flash skins are CSS files that refer to Flex components instead of HTML tags, compiled into SWF format. Through the layout manager users can specify the Flash skin to use for the website. Users can select from a variety of pre-made Flash skins or build and upload their own.

Figure 3: Some Example Flash Skins

There is also a Pay Per Download system that allows a web master to specify files that are available to download and how much they want to charge to download those files. This works by linking to PayPal through the Instant Payment Notification (IPN) API, which provides the user the file to download after successful payment. The user is then emailed a dynamic URL which can be used to download the file up to 24 hours after purchase.

Figure 4: Pay Per Download System

WordPress is a state-of-the-art publishing platform with a focus on aesthetics, web standards, and usability (WordPress.org). Flexsite has the ability to integrate WordPress, so that it can be used to manage the content that the site displays. WordPress supports multiple users with varying account permissions so that site content can be concurrently managed by multiple users. Another benefit of using WordPress is that is accessible to all levels of users since it operates like many familiar Word Processors, and it doesn’t require the download of installation of any software.

Figure 5: WordPress HTML Editor

Flexsite is also used to host its own website, and particular to this installation there is another PayPal integration that allows users to pay for Flexsite hosting services through the website. A graphical user interface is provided which displays all of the service pricing, and allows the user to select a domain name as well as a starting Flash skin for their new website.

Figure 6: Buying Hosting Services

The result is visually appealing and easily maintainable website that doesn’t require the installation of any software, and doesn’t require computer knowledge beyond that of operating a Word Processor. This technology is also accessible to everyone since it cost about the same as regular hosting, but includes the setup, domain name registration, website setup, WordPress setup, and customization.

www.theflexsite.com and www.theflexsite.net

The Flex Tutorial: Part II - Language Concepts and Basics

2008-02-09T21:00:00.000-06:00

The two languages of Flex

The Flex language consists of two flavors: ActionScript 3 and XML. ActionScript is a scripting language based on ECMAScript (JavaScript), used primarily for the development of websites and software using the Adobe Flash Player platform (http://en.wikipedia.org/wiki/ActionScript). The Extensible Markup Language (XML) is a general-purpose markup language, which is classified as an extensible language because it allows its users to define their own elements (http://en.wikipedia.org/wiki/XML). In Flex ActionScript files have the extension ".as" while the XML files have the extension ".mxml."

XML is used in Flex to represent static objects such as a button on a graphical user interface or an HTTP Service. For graphical user interface this provides a fast and efficient way of laying out components. For example consider an application that consists or a single panel and a single button:

Using MXML this would require the following code:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical">
 <mx:Panel width="250" height="200" layout="vertical" horizontalAlign="center" 
 verticalAlign="middle" title="This is a panel">
  <mx:Button label="This is a button"/>
 </mx:Panel>
</mx:Application>

In this Flex (MXML) code you are essentially creating an Application with a vertical layout that contains a Panel with some size, layout, and text attributes. The panel then contains a Button with some text. This is easy to conceptualize since the in the code the application contains the panel, which then contains the button.

Using Java this same equivalent application would require the following code:

import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JButton;
import javax.swing.BorderFactory;
import java.awt.Color;

public class Application extends JFrame {
    
    public static void main(String[] args) {
        JFrame frame = new Application();
        frame.setSize(400,400);
        frame.setVisible(true);
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    }
    
    public Application() {
        JPanel panel = new JPanel();
        panel.setBorder(BorderFactory.createTitledBorder(
            BorderFactory.createLineBorder(Color.black), 
            "This is a panel"));
        JButton button = new JButton("This is a button");
        panel.add(button);
        setContentPane(panel);
    }
}

In the Java code the concept remains the same, that an Application contains a Panel that contains a Button, but the code is not as easy to read (even when ignoring the extra stuff that was thrown in there to close the frame and set the border).

This just serves an example of how much less coding and complexity there is with constructing graphical user interface in MXML than with other object oriented languages. Less complexity means that the application will be easier to maintain, which is extremely important considering that the vast majority of time in the software lifecycle is spent in maintenance. Complexity is not only reduced with Flex and the layout out of graphical user interface components, Flex has numerous other features that do the same across the entire framework.

Though Flex is less complex that doesn't mean that it loses any capability. The genius of Flex is its ability to represent the same conceptual models as in other object oriented languages, only with less code and complexity. Consider the class diagrams of the example Flex and Java applications:

The same relationships are maintained: an application has a panel that has a button. Flex can be used to represent almost any object oriented design exactly. I use the term "almost" because ActionScript 3 currently doesn't support abstract classes.

Flex also has the ability to represent objects using ActionScript, which can be used in place of or in conjunction with MXML. For example you could create a "Person" ActionScript class and use it to represent data in an MXML component. The actual instantiation and value assignments of the "Person" class could then be done in either MXML or ActionScript.

The following is an ActionScript representation of a "Person" class:

package example.data
{
 public class Person
 {
  public var firstName:String;
  public var lastName:String;
  public var favoriteFood:String;
 }
}

Inside XML this class could be instantiated (assuming the class was included as "ex") like so:

<ex:Person id="person" firstName="John" lastName="Valentino" favoriteFood="Steak"/>

Inside of ActionScript this class could be instantiated like the following:

var person:Person = new Person();
person.firstName = "John";
person.lastName = "Valentino";
person.favoriteFood = "Steak";

What the heck are you doing? Where are the getters and setters?

Getters and setters are something that I feel have gotten a bit out of hand in object-oriented programming. The purpose of using getters and setters is to decouple objects and to provide hidden functionality when assigning values to attributes of objects. For example if the "Person" class contained an instance of "Person" called "child" you probably wouldn't want to make it (and all of its attributes) publicly accessible to prevent coupling. An example of hidden functionality would be if you wanted to check modify special characters in the first and last names when setting their values.

No matter how you or I feel about getters and setters and their under usage or over usage, Flex provides its own special functions for getters and setters. This allows classes to define methods to handle the getting and setting of attributes within classes, but those functions can be called (by lazy programmers like myself) as if they were publicly accessible attributes.

Consider the "Person" class redone to use the special get and set functions:

package example.data
{
 public class PersonII
 {
  private var _firstName:String, _lastName:String, 
        _favoriteFood:String;
  
  public function get firstName():String
  {
   return _firstName;
  }
  
  public function get lastName():String
  {
   return _lastName;
  }
  
  public function get favoriteFood():String
  {
   return _favoriteFood;
  }
  
  public function set firstName(value:String):void
  {
   _firstName = value;
  }
  
  public function set lastName(value:String):void
  {
   _lastName = value;
  }
  
  public function set favoriteFood(value:String):void
  {
   _favoriteFood = value;
  } 
 }
}

These special get and set functions are what allow these functions to be called as if they were attributes. Using this new person classes the example instantiations in MXML and ActionScript to not need to be changed; they work the same as before.

Inside XML this class could be instantiated (assuming the class was included as "ex") like so:

<ex:PersonII id="person" firstName="John" lastName="Valentino" favoriteFood="Steak"/>

Inside of ActionScript this class could be instantiated like the following:

var person:PersonII = new PersonII();
person.firstName = "John";
person.lastName = "Valentino";
person.favoriteFood = "Steak";

Bringing MXML and ActionScript together

One of the most important concepts in Flex is that every object has 2 aspects: the XML and the ActionScript. In a constant effort to eliminate complexity I have found that XML is preferable to ActionScript in most situations. A good rule of thumb to know when to use what is to ask yourself the question "Can I do this in MXML?" If the answer is no than you can go with ActionScript, otherwise stick with MXML to cut down on code size and complexity. You must exercise common sense though; if you find yourself having to do something that seems unintuitive in MXML than you should probably use ActionScript.

Typically in graphical user interface components ActionScript is used within MXML components. This allows graphical user interfaces to be statically laid out in MXML, and then use ActionScript to handle complex logic. For example consider the original example application that has been modified to use ActionScript to display a message when a button is pressed:

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical">

 <mx:Script>
  <![CDATA[
  import mx.controls.Alert;

  private function buttonPressed():void
  {
   Alert.show("The button was pressed");
  }
  ]]>
 </mx:Script>

 <mx:Panel width="250" height="200" layout="vertical" horizontalAlign="center" 
  verticalAlign="middle" title="This is a panel">
  <mx:Button label="This is a button" click="buttonPressed()"/>
 </mx:Panel>
</mx:Application>

Now when the button is pressed the following message is displayed:

Binding

One of the most important features of Flex is the ability to bind objects to components. This is basically instant model-view-controller without having to use listeners or worry about observers and observables. A common need in programming graphical user interfaces is the ability to display some type of information to the user through some sort of component, that is then updated through some event. For example consider a series of text fields that display information about a person:

The following code shows how the data is bound to different fields within the graphical user interface:

<?xml version="1.0" encoding="utf-8"?>
<mx:Panel xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical" 
 title="Person Panel" horizontalAlign="center">

 <mx:Script>
  <![CDATA[

  [Bindable]
  private var firstName:String = new String();
  [Bindable]
  private var lastName:String = new String();
  [Bindable]
  private var favoriteFood:String = new String();

  private function john():void
  {
   firstName = "John";
   lastName = "Valentino II";
   favoriteFood = "Steak";
  }

  private function sarah():void
  {
   firstName = "Sarah";
   lastName = "Valentino";
   favoriteFood = "Pizza";
  }

  ]]>
 </mx:Script>
 <mx:Form width="100%">
  <mx:FormItem label="First Name:">
   <mx:TextInput text="{firstName}"/>
  </mx:FormItem>
  <mx:FormItem label="Last Name:">
   <mx:TextInput text="{lastName}"/>
  </mx:FormItem>
  <mx:FormItem label="Favorite Food:">
   <mx:TextInput text="{favoriteFood}"/>
  </mx:FormItem>
 </mx:Form>
 <mx:HBox>
  <mx:Button label="John" click="john()"/>
  <mx:Button label="Sarah" click="sarah()"/>
 </mx:HBox>

</mx:Panel>

The variables firstName, lastName, and favoriteFood, are made "Bindable" so that when ever they are changed anything that is referencing them is also changed. This means that since those variables are referenced by the TextInputs, when those variables are changed what is displayed in those TextInputs will also change.

For example the following is displayed when the "John" button is pressed:

The following is displayed when the "Sarah" button is pressed:

It should also be noted that if you declare instances in MXML that are automatically bindable, so the following example retains the same functionality:

<?xml version="1.0" encoding="utf-8"?>
<mx:Panel xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical" 
 title="Person Panel" horizontalAlign="center">
 
 <mx:String id="firstName"/>
 <mx:String id="lastName"/>
 <mx:String id="favoriteFood"/>

 <mx:Script>
  <![CDATA[

  private function john():void
  {
   firstName = "John";
   lastName = "Valentino II";
   favoriteFood = "Steak";
  }

  private function sarah():void
  {
   firstName = "Sarah";
   lastName = "Valentino";
   favoriteFood = "Pizza";
  }

  ]]>
 </mx:Script>
 <mx:Form width="100%">
  <mx:FormItem label="First Name:">
   <mx:TextInput text="{firstName}"/>
  </mx:FormItem>
  <mx:FormItem label="Last Name:">
   <mx:TextInput text="{lastName}"/>
  </mx:FormItem>
  <mx:FormItem label="Favorite Food:">
   <mx:TextInput text="{favoriteFood}"/>
  </mx:FormItem>
 </mx:Form>
 <mx:HBox>
  <mx:Button label="John" click="john()"/>
  <mx:Button label="Sarah" click="sarah()"/>
 </mx:HBox>

</mx:Panel>

If you want to use binding with a custom class however, such as with the Person class, you have to declare the class itself or its individual attributes as bindable. This allowing the class to be bound, but the instance still has to be declared as bindable. For example consider the bindable Person class:

package example.data
{
 [Bindable]
 public class Person
 {
  public var firstName:String;
  public var lastName:String;
  public var favoriteFood:String;
 }
}

To use the bindable Person class in the application example the Person instance has to be marked as "Bindable." The following is the application redone to use the Person class:

<?xml version="1.0" encoding="utf-8"?>
<mx:Panel xmlns:mx="http://www.adobe.com/2006/mxml" layout="vertical" 
 title="Person Panel" horizontalAlign="center">

 <mx:Script>
  <![CDATA[
  import example.data.Person;
  
  [Bindable]
  private var person:Person = new Person();

  private function john():void
  {
   person.firstName = "John";
   person.lastName = "Valentino II";
   person.favoriteFood = "Steak";
  }

  private function sarah():void
  {
   person.firstName = "Sarah";
   person.lastName = "Valentino";
   person.favoriteFood = "Pizza";
  }

  ]]>
 </mx:Script>
 <mx:Form width="100%">
  <mx:FormItem label="First Name:">
   <mx:TextInput text="{person.firstName}"/>
  </mx:FormItem>
  <mx:FormItem label="Last Name:">
   <mx:TextInput text="{person.lastName}"/>
  </mx:FormItem>
  <mx:FormItem label="Favorite Food:">
   <mx:TextInput text="{person.favoriteFood}"/>
  </mx:FormItem>
 </mx:Form>
 <mx:HBox>
  <mx:Button label="John" click="john()"/>
  <mx:Button label="Sarah" click="sarah()"/>
 </mx:HBox>

</mx:Panel>

The Flex Tutorial: Part I - Setup and Organization

2008-01-09T22:13:00.000-06:00

What is this tutorial?

"Flex development" refers to the use of the Adobe Flex to develop web applications that are deployed in the Adobe Flash runtime. The term "Enterprise" in the context of software development is usually used to indicate that something is large-scale, supports a large user base, and consists of many different systems or services. So I use the term "Flex Enterprise Development" to refer to large-scale flex web applications with emphasis on taking advantage of Flex features such as Data Binding and Web Services, and the use of architectures such as SOA, Three-Tier, and MVC. In particular Data Binding can be used to easily implement a MVC framework in an application with little to no extra code, and on a larger scale a combination of Three-Tier and SOA can be used to organize components and make a system easily scalable. This tutorial intends to explain how to setup a development environment, show the important features of the Flex language, and show how to organize and implement small to large-scale systems.

Setting up a Flex Development Environment

In terms of treating web systems as three-tier architectures, meaning that on a high level they can be divided into presentation, service, and data tiers, Flex applications reside in the presentation tier. This means that for servicing it is required that server side scripting languages are used. Flex is not dependent on a single server side scripting language though, since Flex provides the ability to work with data formats such as WSDL, SOAP, Remote Objects, and HTTP Services. I tend to prefer using HTTP Services since is is just plain XML that you can define yourself without headers or envelopes, and is not specific to scripting language remote object technologies.

Adobe provides a free SDK for Flex, but the Flex Builder development environment (http://www.adobe.com/products/flex/flexbuilder/) is a huge time saver since it allows you to rapidly develop GUIs by dragging and dropping pre-made components and editing them accordingly. The first step for setting up a development environment is therefore to download and install Flex Builder.

The next step is to install and configure some sort of web service so that you can develop web services using a server side scripting language. You should also be aware that this choice will also affect which database you can use if you choose to use one. You should take into account that if you intend to deploy your system on someone else's hardware, meaning a web hosting service, you want to setup a similar environment. You should also be aware that most hosting services such as GoDaddy.com provide two options: a Windows (IIS) hosting plan that includes ASP/.NET with MS SQL and MS Access, and a Linux (Apache) hosting plan that includes PHP with MySQL. If you want to use something else like J2EE or ColdFusion you will need use a less traditional web hosting service or use your own hardware.

Personally I run IIS on port 80, Apache on port 8000, and JAS (Java Application Server for J2EE) on port 8080 on my Windows XP MCE laptop. I would like to point out though that setting up IIS on Windows XP MCE was difficult because of a lot of missing components and other issues that can be found with a quick web search. Apache is both free and easy to install, and I actually pointed its PHP services to the same one that I use for IIS, but there is really no need to have more than one web service and server side scripting language unless you are using more than one.

If you want to use IIS you have to be running Windows XP Pro or MCE, and can get it by putting in the Windows Installation CD and choosing to add IIS as an additional component. If you want to use Apache you can download it from the Apache website at http://httpd.apache.org/. IIS comes .NET and ASP ready which cannot used at all on Apache, but both Apache and IIS can be configured to use PHP. PHP can be downloaded for both Windows and Linux at http://www.php.net/downloads.php, along with instructions for how to configure on both IIS and Apache. If you want to use Java you can ether download JAS from http://java.sun.com/ along with the NetBeans IDE, or you can download Tomcat from http://tomcat.apache.org/.

For PHP, Java, and JSP though I prefer to use the Eclipse IDE. The Eclipse IDE can be downloaded from http://www.eclipse.org/downloads/, and comes with the ability to do Java. If you want to do PHP you can download a free plug-in from http://www.zend.com/en/community/pdt, and if you want to do JSP I recommend paying for the MyEclipse plug-ins at http://www.myeclipseide.com/.

Once you have decided on web services and language you will end up with with two IDEs: FlexBuilder and one dedicated to your server side scripting language of choice.

Setting up a Project

You want to create a project directory (through FlexBuilder) in your web root directory. Your web root directory is the directory setup to run web services you your computer. If you are running IIS the default web root directory is C:\inetpub\wwwroot, on Apache Windows the default directory is C:\Program Files\Apache Software Foundation\Apache2.2\htdocs, and on Apache Linux the default directory is /var/www/html/. All of these locations correspond to http://localhost/ using the web service that you are running.

You want to layout your project directory as follows:

- <Project Directory>: HTML page and scripts that execute your Flash SWF.
 - src: Flex source code, MXML and AS
 - html-template: templates for generating the HTML page used to run the Flash
 - service: service side scripting used to represent web services
 - data: data structure representations used by the server side scripting
 - settings: miscellaneous files for static content, for example CSS, a web services list...

It should also be noted that for larger projects the web servicing (the server side scripting in the service and data directories) may be located on another machine. In order for your Flex application to access content in an external location on the web, the root web directory of the server

This can be initiated in Flex Builder by specifying your root web directory as the workspace and creating a new project there. Then in the "New Flex Project" window specify the main source folder as "src" and the output folder as blank.

Once the project is created you also have to make sure that the runtime settings point to the HTTP location of the SWF file instead of the File location. This is done by changing the "URL or path to launch" in the Run Dialog for the project.

Laying Out the GUI

Once the project is created there is a single MXML file in your "src" directory, which is the root of the application. From this point it is best to divide up your application into packages based on sub-systems, presentation tier components, and classes used to represents data. It should also be noted that for larger applications you may want to separately group classes related to HTTP and other web services. For example if you had an application with several detailed panels, you may want to break each detailed panel into a separate class. The following is an example directory structure a flex application that divides it components by the following groups: Core Components (core), User Management Components (user), and Example (example)

- src: This is the source code directory
 - core: custom components used by all packages
  * MyFancyButton.mxml
 - user: components related for managing user accounts
  - data: data structures related to user components
   * UserRecord.as
  - interfaces: interfaces used for communicating with the user components
   * LoginInterface.as
   * ManageAccountInterface.as
  * LoginPanel.mxml
  * ManageAccountPanel.mxml
 - example: general components used for demonstrating Flex abilities
  - data: data structures used for the example components
   * CDRecord.as
  * WelcomePanel.mxml
 * MyProject.mxml

For example consider layout our a basic application that will display only the welcome screen as a custom component. First you would layout the entire GUI minus the welcome panel, which may look something like the following:

Next make a simple component named "WelcomePanel.mxml" in the "example" directory that inherits from "Panel."

Using State Transitions

Now that the application is laid out and there is a custom component that can be used, a state transition can be used to add the custom component to the application. In the "MyProject.mxml" component there is a "States" box that has a single state listed, which is the base state.

If you right-click on the base state and select the option to create a "New State," you can create a state called "WelcomeState" which can be used to display the welcome panel.

With the "WelcomeState" selected you can then drag and drop the "WelcomePanel" onto the "MyProject" component:

If you then reselect the base state the welcome panel will disappear, because the application goes back to its original state. If in the base state you select the welcome button from the menu panel and specify the "click" attribute to be "currentState='WelcomeState", when the welcome button is pressed the application will move from the base state to the "WelcomeState."

Flex-Based Message Board

2007-10-25T20:37:00.001-06:00

A while ago I finally got fed up with a hosting service I was using for a particular product's web registration service, so I moved everything to a new host and decided to make some needed changes. One change in particular was the message board. I was using phpbb and was spending too much time battling spammers. I originally had it to where anyone could post, because most people with legit questions prefer not to have to create accounts. This of course was fine until the spammers found it, so I turned on the all the validation stuff and people just stopped using the board entirely (and emailing me questions). With the new host I was in need of a new message board, so I decided to make one myself using Flex (and PHP5). This is the current board: http://www.vdrillpro.net/flexboard/

My concept is to allow people to post semi-anonymously by allowing users to use any name they want, and all they have to do it leave a valid email address that no one can see (but admins). Users then have to enter a validation code, and are then sent an email in which they have to click on the provided link to move their message out of the queue and on to the board to be seen by others. I am hoping that the anonymous posting will encourage more legit posters, and that with the email validation it will slow down the spam but not be too much of a pain. I also built in the ability to ban email addresses by regular expression, which allows me to use keywords and do stuff like ban all .info, .biz, .cn, and .ru addresses.

The layout is also different, displaying topics and comments in grid form to give it more of a desktop application feel:

Topic view allows the user to see the full content of the post, and all of the replies in grid form:

Replies also allow rich text with links:

An validation code must also be entered before a post is queued:

An email is then sent to the address that was entered containing a link. When the user clicks on that link the post is out of the queue and can then be seen on the message board.

Introduction to the Spring Framework

2007-10-21T16:37:00.000-06:00

What is the Spring Framework?

According to SpringFramework.org:

Spring provides a light-weight solution for building enterprise-ready applications, while still supporting the possibility of using declarative transaction management, remote access to your logic using RMI or web services, and various options for persisting your data to a database [1].

More specifically Spring is a way of using regular Java Beans as Enterprise JavaBeans without involving the complexity of J2EE. JavaBeans encapsulate many objects into a single object (the bean), so that the bean can be passed around rather than the individual objects [2]. The Enterprise JavaBean (EJB) is a managed, server-side component architecture for modular construction of enterprise applications [3], which requires a J2EE container/application server. Java 2 Enterprise Edition (J2EE) is a combination of libraries and useful technologies for the Java programming language, that are used to develop and deploy enterprise applications using modular components using a Java application server.

Spring is more than just JavaBeans though, it is a framework that contains other frameworks, which are designed to independently work well with other frameworks. These frameworks or modules are typically divided into the following:

Inversion of Control Container (Core Container)
Context Module
Aspect Oriented Programming (AOP) Module
Data Access Object (DAO) Module
Object-Relational Mapping (ORM) Module
Web Module
Model-View-Controller (MVC) Framework

For the purposes of introduction though, it is best to start with the basics of the Inversion Control Container. Central in the Spring Framework is its Inversion of Control Container that provides a consistent means of configuring and managing Java objects. This container is also known as BeanFactory, ApplicationContext or Core container [4]. Inversion of Control is a concept in which the control flow is inverted compared to the traditional interaction model expressed in imperative style by a series of procedure calls. Thus, instead of the programmer specifying, by the means of function calls, a series of events to happen during the lifetime of a programme, they would rather register desired responses to particular happenings, and then let some external entities take over the control over the precise order and set of events to happen [5].

Where do I get the Spring Framework and how do I install it?

The framework is a series of JAR files, which can be downloaded from http://www.springframework.org/download for fee. To install it all you really have to do is extract the downloaded compressed file to some location on your computer, and then reference the appropriate JARs in a Java project. If you download the full distribution you will get the following directories, which are specified "RELEASE INFO" section of the "readme.txt" included with the distribution:

dist - contains the Spring distribution jar files, as well as a zip of all Java source files
docs - contains the Spring reference documentation in PDF and HTML format, as well as the complete API javadocs
lib - contains all third-party libraries needed for building the framework and/or running the samples
src - contains the general Java source files for the framework
mock - contains the general Java source files for Spring's mock and test classes
test - contains the general Java source files for Spring's test suite
tiger/src - contains the JDK-1.5-specific Java source files for the framework
tiger/test - contains the JDK-1.5-specific Java source files for Spring's test suite
aspectj/src - contains the AspectJ-specific source files for the framework
aspectj/test - contains the AspectJ-specific source files for Spring's test suite
samples - contains various demo applications and showcases

How do I use the Spring Framework?

Considering that there are many sub-frameworks in Spring, it is easiest to start with an example that demonstrates the core basics of the Inversion of Control Container. There are many available Spring Tutorials that can be found in books and on the internet. So far the best example that I have been able to find is the Inversion of Control example from DevelopersBook.com, in which you don't directly connect your components and services together in code but describe which services are needed by which components in a configuration file [6]. I have taken the basic example and modified it to include comments and a few other properties, with the intention for later use in an Aspect Oriented Programming example. Consider a simple application framework that provides JavaBean that is used to greet as specified person using a "Greeting Service," and say good-bye to that person using a "Leaving Service." This requires the following classes, interfaces, and XML configuration:

HelloBean.java - This is the bean that is used to encapsulate the "Greeting Service" and "Leaving Service."
HelloBean.xml - This is the Spring XML configuration file that is used to specify the instance and relationships of the involved object.
GreetingService.java (implementing GreetingInterface.java) - This is the service that is used to generate the greeting message.
LeavingService.java (implementing LeavingInterface.java) - This is the service that is used to generate the leaving message.
HelloClient.java - This is the main class that uses the Inversion of Control Container with the Spring XML configuration file to get the specified instance of the HelloBean.

The following is the project directory structure, which shows the location of the Java source code files, the related class files, and the Spring XML configuration file:

bin
- springapp
  - service
    - interfaces
      - GreetingInterface.class
      - LeavingInterface.class
    - GreetingService.class
    - LeavingService.class
  - bean
    - HelloBean.class
  - HelloClient.class
- HelloBean.xml
src
- springapp
  - service
    - interfaces
      - GreetingInterface.java
      - LeavingInterface.java
    - GreetingService.java
    - LeavingService.java
  - bean
    - HelloBean.java
  - HelloClient.java

Architecturally the following class diagram shows how all of the objects relate to one another within the "springapp" package:

The "HelloClient" uses the Spring XML configuration file "HelloBean.xml" with the Inversion of Control Container to create the specified instance of the "HelloBean." The "HelloBean" encapsulates the objects and functionality of the "GreetingService" and "LeavingService."

For this particular application, the following libraries from the Spring Framework are required:

<spring-install>\lib\jakarta-commons\*.jar
<spring-install>\lib\log4j\lg4j-1.2.14.jar
<spring-install>\dist\spring.jar

GreetingService.java

This is an object that is instantiated by the "HelloBean" in order to generate the greeting message. The actual instantiation though is specified in the "HelloBean.xml" file, which is handled by the Inversion of Control Container or BeanFactory. This class only has one property, which is the variable "greetingMessage." In order for the BeanFactory to be able to allow the setting of the value of this variable the class must provide a "setter" method. In the case of the "GreetingService," it has a "setGreetingMessage" method that can used by the BeanFactory to set the value of the "greetingMessage" variable.

package springapp.service;

import springapp.service.interfaces.GreetingInterface;

/**
 * <p>Title: Greeting Service</p>
 * 
 * <p>Description: This is the service that is used to generate
 * the greeting message.</p>
 * <p>Taken and modified from Code Listing 1:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class GreetingService implements GreetingInterface {

 /** This is greeting message that is displayed.*/
 private String greetingMessage;

 /**
  * Returns the "Hello" message for the person of the given name.
  * @param name
  * @return String
  */
 public String sayHello(String name) {
  return greetingMessage + " " + name + ".";
 }

 /**
  * Sets the greeting message
  * @param message
  */
 public void setGreetingMessage(String message) {
  this.greetingMessage = message;
 }

}

GreetingInterface.java

The methods required to be implemented by the "GreetingService" that are public and are not "getter" or "setter" methods are specified in this interface. This is because it is considered good practice to separately specify those methods in an implemented interface.

package springapp.service.interfaces;

/**
 * <p>Title: Greeting Interface</p>
 * 
 * <p>Description: Representing the methods of the Greeting Service.</p>
 * <p>Taken and modified from Code Listing 1:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public interface GreetingInterface {

 /**
  * Returns the "Hello" message for the person of the given name.
  * @param name
  * @return String
  */
 public String sayHello(String name);
}

LeavingService.java

This is an object that is instantiated by the "HelloBean" in order to generate the leaving message. The actual instantiation though is specified in the "HelloBean.xml" file, which is handled by the Inversion of Control Container or BeanFactory. This class only has one property, which is the variable "leavingMessage." In order for the BeanFactory to be able to allow the setting of the value of this variable the class must provide a "setter" method. In the case of the "LeavingService," it has a "setLeavingMessage" method that can used by the BeanFactory to set the value of the "leavingMessage" variable.

package springapp.service;

import springapp.service.interfaces.LeavingInterface;

/**
 * <p>Title: Leaving Service</p>
 * 
 * <p>Description: This is the service that is used to generate
 * the leaving message.</p>
 * <p>Taken and modified from Code Listing 2:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class LeavingService implements LeavingInterface {

 /** The good-bye message to be displayed. */
 private String leavingMessage;

 /**
  * Returns the "Good-Bye" message for the person of the given name
  * @param name
  * @return String
  */
 public String sayGoodBye(String name) {
  return leavingMessage + " " + name + ".";
 }

 /**
  * Sets the good-bye message
  * @param message
  */
 public void setLeavingMessage(String message) {
  this.leavingMessage = message;
 }

}

LeavingInterface.java

The methods required to be implemented by the "LeavingService" that are public and are not "getter" or "setter" methods are specified in this interface.

package springapp.service.interfaces;

/**
 * <p>Title: Leaving Interface</p>
 * 
 * <p>Required methods for the Leaving Service.</p>
 * <p>Taken and modified from Code Listing 2:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 * 
 * @author John Valentino II
 *
 */
public interface LeavingInterface {

 /**
  * Returns the "Good-Bye" message for the person of the given name
  * @param name
  * @return String
  */
 public String sayGoodBye(String name);

}

HelloBean.java

This is the JavaBean that is used to encapsulate the services of this example application. Since this class has three properties, it specifies three "setter" methods to allow the setting of instances or values by the BeanFactory or other parent classes.

package springapp.bean;

import springapp.service.GreetingService;
import springapp.service.LeavingService;

/**
 * <p>Title: Hello Bean</p>
 * 
 * <p>Description: Represents the Hello Bean.</p>
 * <p>Taken and modified from Code Listing 4:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class HelloBean {

 /** Represents the service used to generate a greeting */
 private GreetingService greetingService;
 /** Represents the service used to generate a good-bye message */
 private LeavingService leavingService;
 /** Represents the name of the person to meet or leave */
 private String personName;

 /**
  * Sets the name of the person
  * @param name
  */
 public void setPersonName(String name) {
  this.personName = name;
 }

 /**
  * Sets the instance of the greeting service
  * @param service
  */
 public void setGreetingService(GreetingService service) {
  this.greetingService = service;
 }

 /**
  * Set the instance of the leaving service
  * @param service
  */
 public void setLeavingService(LeavingService service) {
  this.leavingService = service;
 }

 /**
  * Generates the greeting message
  * @return
  */
 public String greet() {
  return greetingService.sayHello(personName);
 }

 /**
  * Generates the leaving message
  * @return
  */
 public String leave() {
  return leavingService.sayGoodBye(personName);
 }

}

HelloClient.java

This is the main class of the application that uses the "HelloBean.xml" with the Inversion of Control Container to create the instance of the "HelloBean" and call is available methods.

package springapp;

import springapp.bean.HelloBean;
import org.springframework.context.support.ClassPathXmlApplicationContext;

/**
 * <p>Title: Hello Client</p>
 * 
 * <p>Description: Represents a program that uses the HelloBean.</p>
 * <p>Taken and modified from Code Listing 5:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class HelloClient {

 /**
  * @param args
  */
 public static void main(String[] args) {

  //Get the instance of the HelloBean as specified in HelloBean.xml
  ClassPathXmlApplicationContext appContext = 
   new ClassPathXmlApplicationContext(new String[] { "HelloBean.xml" });
  HelloBean helloBean = (HelloBean) appContext.getBean("helloBean");

  System.out.println(helloBean.greet());
  System.out.println(helloBean.leave());

 }

}

HelloBean.xml

This is the Spring XML configuration file that is used to specify the instances and properties of the bean classes within the application.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" 
"http://www.springframework.org/dtd/spring-beans.dtd">

<beans>
 <!-- Creates an instance of the "HelloBean" class -->
 <bean id="helloBean" class="springapp.bean.HelloBean">
  <!-- Sets the value of "personName" by
       calling "setPersonName" -->
  <property name="personName">
   <value>Sarah</value>
  </property>
  <!-- Sets the instance of "greetingService" by
       calling "setGreetingService" -->
  <property name="greetingService"> 
   <ref bean="greetingService" />
  </property>
  <!-- Sets the instance of "leavingService" by
       calling "setLeavingService" -->
  <property name="leavingService">
   <ref bean="leavingService" /> 
  </property> 
 </bean>
 <!-- Creates an instance of the "GreetingService" -->
 <bean id="greetingService" class="springapp.service.GreetingService">
  <!-- Sets the value of "greetingMessage" by 
         calling "setGreetingMessage" -->
  <property name="greetingMessage">
   <value>Hello</value>
  </property>
 </bean>
 <!-- Creates an instance of the "LeavingService" -->
 <bean id="leavingService" class="springapp.service.LeavingService">
  <!-- Sets the value of "leavingMessage" by
       calling "setLeavingMessage" -->
  <property name="leavingMessage">
   <value>Good-bye</value>
  </property>
 </bean>
</beans>

Output

The program runs and generates the following output:

Hello Sarah.
Good-bye Sarah.

This output is to be expected as the Spring XML configuration file specifies that the "HelloBean" instance has the specified "personName," an instance of the "GreetingService," and an instance of the "LeavingService." Both service instances are also given specified "greetingMessage" and "leavingMessage" values.

Links

Spring Framework IoC Tutorial: http://www.developersbook.com/spring/spring-tutorials
Spring Framework MVC Application Step by Step: http://www.springframework.org/docs/MVC-step-by-step/Spring-MVC-step-by-step-Part-1.html
Spring Framework Application Tutorial: http://www.roseindia.net/spring/springpart2.shtml
Spring Framework Documentation: http://static.springframework.org/spring/docs/2.0.x/reference/index.html
How to create a bean descriptor XML file: http://www.visualbuilder.com/java/spring/tutorial/pageorder/8/
Inversion of Control: http://en.wikipedia.org/wiki/Inversion_of_control
Aspect-Oriented Programming: http://en.wikipedia.org/wiki/Aspect-oriented_programming
Factory Method Pattern: http://en.wikipedia.org/wiki/Factory_design_pattern

References

SpringFramework.org, Documentation Preface, http://static.springframework.org/spring/docs/2.0.x/reference/preface.html
Wikipedia, JavaBean, http://en.wikipedia.org/wiki/JavaBeans
Wikipedia, Enterprise JavaBean, http://en.wikipedia.org/wiki/Ejb
Wikipedia, Spring Framework, http://en.wikipedia.org/wiki/Spring_framework
Wikipedia, Inversion of Control, http://en.wikipedia.org/wiki/Inversion_of_Control
DevlopersBook.com, Spring Framework Tutorial, http://www.developersbook.com/spring/spring-tutorials/spring-tutorials.php

Instant Bach goes Open Source

2007-05-11T15:31:00.000-06:00

Introduction

The program "Instant Bach" was made as a project for an unnamed computer science class, which was when I was a sophomore computer science major, and a senior music performance and music composition double-major. Needless to say at the time of this program's creation I knew far more about musical theory than I did computer science. When I approached the professor of this class with my project idea his response was that "other people have tried stuff similar for this class and failed, and the same will happen to you." Despite this opinion I remained confident that a four-part harmony writing program could be made using the skills I currently had available, so I went ahead and made the program for my project. I have recently decided to resurrect this program to expand its capabilities as an open source project (http://sourceforge.net/projects/instantbach).

The original program was based of of static rules which limits it to a particular number of possible progressions:

My most immediate goal for this project is to move all of the rules of voice leading, doubling, chord structure, and progression to an external rules file, so that the program can be adjusted to work beyond just being able to voice major key progressions.

Purpose

The first two years of undergraduate music theory are spent learning and doing mostly one thing: four-part harmony writing. The purpose of this program is to be able to do four part harmony writing, with or without a user specified progression, according to all the rules of four part harmony writing, and have the ability to transpose the result into any key.

Musical Intervals

The distance between any two musical notes is referred to as an interval, and is represented by a two-character symbol. The first character denotes the quality, being P for perfect, A for augmented, D for diminished, m for minor, and M for major. The second character denotes the melodic distance with a number of how many lines and spaces exist between the two notes of the interval. Major and minor refer to whether an interval occurs within a natural minor scale or within a major scale. Augmented and diminished are used for intervals that do not exist in the major or natural minor scales, augmented being greater than, and diminished being less than. The following is a chart of the most common musical intervals:

Tertian Harmony

Tertian Harmony refers to simultaneously sounding pitches in which each of the pitches are separated by the distance of major or minor third, in at least one permutation of the order of notes. Tertian harmonies consisting of three distinct notes are referred to as triads, and harmonies consisting of four distinct notes are referred to as seventh chords. A triad can have four different classifications depending on the interval relationship between the notes in the triad. These different types of triads are called major, minor, augmented, and diminished. Seventh chords can be major-major, dominant, minor, half-diminished, or fully diminished.

Progression

For any given scale, for example a C Major scale (C, D, E, F, G, A, B), each of the pitches in that scale represent the root note of a tertian chord of some type. Each of these harmonies are referred to as chords and are represented by a roman numeral. The roman numeral indicates the quality of a chord (major, minor, augmented, diminished), the position of the chord inside the key, and which note is on the bottom, (the root), of that chord.

The first note of a chord is known as the root, the second is the third, the third is the fifth, and if there is a fourth note it is called the seventh. Whenever a chord cannot be spelled in thirds, it is in an inversion, meaning that a note other then the root is on the bottom of the chord. An inversion is denoted by one number on top of another and is written to the right of the chord symbol, but I will write them as fractions for simplicity. In a triad, when the third is on the bottom of the chord it is in first inversion and the symbol that is used is 6. When the fifth is on the bottom of a triad it is in second inversion and is written as 6/4. Seventh chords can have three inversions, with the third, fifth, and seventh on the bottom, denoted 6/5, 4/3, and 2.

Rules of Progression

The order in which chords can occur have multiple rules, the easiest rule being that I can move to anything, but rather than explain the progression behavior with words, I will use a picture of the progression graph, where each chord symbol is a vertex, and all edges are un-weighted.

Other Chord Symbols

As shown in the graph, there are other chord symbols that have not yet been described. They are the Neopolitan Sixth Chord (N6), the French Augmented Sixth Chord (Fr+6), and the secondary dominant (V/x). These chords incorporate notes that are outside of the given key, and have strict rules for which chord they can lead to. The N6 and Fr+6 must go to V, and secondary dominants must lead to chord symbol on the bottom of the fraction. Secondary dominants are considered to be a borrowed V from another key, and therefore must progress to the I of the key from which they were borrowed.

Four part Harmony Writing

Four-part harmony writing is when a progression is developed and the notes in the chords of that progression are assigned to four voices, the bass, tenor, alto, and soprano. Each voice has to stay within a particular range and is limited to the distance which the voice can move between chords. Each note has rules for motion and each chord has rules for its resolution. Because most of the progression consists of triads, and there are four voices, a member of the chord has to be doubled, which there are also rules for.

Rules of Doubling

I, ii, IV, iii, and vi : The root and third can be doubled.
V and V/x : The root can be doubled.
V7 and V7/x : no doubling is needed or the root can be doubled and the fifth omitted.
viio : The third can be doubled.
vii07 : No doubling is needed.
N6 : The third can be doubled.
Fr+6 : No doubling is needed.

Rules of Resolution

I, ii, IV, and vi : None.
iii : If the fifth is in the soprano or bass it must resolve up by a m2. If the fifth is in the tenor or alto it must either resolve up a m2 or move down a M3.
V and V/x : If the third is in the soprano or bass it must resolve up by a m2. If the third is in the tenor or alto it must either resolve up a m2 or move down a M3.
V7 and V7/x : The third must resolve as the same as in the V and V/x and the seventh must move down by a m2.
viio : If the root is in the soprano or bass it must resolve up by a m2. If the root is in the tenor or alto it must either resolve up a m2 or move down a M3. If the chord which the viio is going to is a V, then the leading tone (the seventh degree of the scale) is already present and there are no rules of resolution.
vii07 : The root must resolve the same as in the viio, and even if the next chord is V, the seventh must move down by a M2.
N6 : The root must move down by a D3.
Fr+6 : The A6 (or its inversion d3) interval in the chord must resolve to P8 or P1.

Rules of Individual Voice Motion

Bass: The bass generally skips around more than all of the other voices and does not have any specific restrictions.
Tenor and Alto: Both these voices generally move very little and rarely move more then the interval of a third.
Soprano: This voice moves more then the tenor and alto but is usually limited to skips of less then a P4.

Range of Voices in Relation to Each Other

The distance between the tenor and the bass can never exceed a P8+ M3, while the distance between all other voices must remain within a P8.

Cross-Voicing

All voices must always be in order from lowest to highest: bass, tenor, alto, and soprano. If a voice is ever out of that order it is considered cross-voicing, which is not allowed. Cross-voicing can also occur in the motion between two chords.

Illegal Parallel Motion

If two voices move the same interval in the same direction they are in parallel motion. The parallel motions of a P8, P5, or P1 are illegal parallel motions. These are called parallel octaves, fifths, and unisons.

Programming the Progression

The progression for the program can be created two ways, either the user can select the individual chords, or the computer can generate the progression itself. Harmonic progression can be represented as an unweighted graph, in order to determine whether a specified progression is valid. The main problem with this approach is that if the computer were to generate its own progression, it would have to end at a specific point with a cadence. (A cadence is a series of chords which end a musical idea or statement). The computer is able to do two of the most common cadences, an imperfect authentic cadence (IAC), and a perfect authentic cadence (PAC). Each of the cadences consists of a V then I, but in the PAC the soprano shares the root with the bass. The problem is that if you have n chords and start with a random chord, and randomly move through the graph until n-2, then cadence, there is a good chance that the chord at n-3 will not be able to go to the chord at n-2. This would mean that the progression would have to be re-done. To prevent this problem, I built the graph backwards, and the progression is formed backwards as well. This way the progression never runs itself into a dead-end because it always starts with I, V, and then the random chords, so when the progression reaches n, the chord at n will always exist because there are no verticies in the graph that do not have at least one adjacent vertex.

Turning the Progression into Musical Notes

After the progression has been created, the computer needs a way to assign notes of all chords in the progression to the four voices. To assign notes to the four voices the computer must first generate all possible notes each voice can have. This is done by using a recursive method that when given the starting pitch of a chord, and the symbol of that chord, builds a chord consisting of every possible note in that chord. Because rules of voice leading depend on individual chords and the position of individual notes within that chord, each note in a chord maintains whether a given note is a root, third, fifth, or seventh.

The Chord as a Data Structure

The main object of this program is the chord, which consists of four voices and a chord symbol, so the obvious main data structure for the program is a Chord object, consisting of four voices and a chord symbol. A Chord object also contains methods which allow the individual voices to be viewed as either the note or the position of that note within the chord, as well as a method to obtain the chord symbol. This allows other methods to deal with notes arithmetically and as the position of that note within the chord for voice leading purposes.

Creating All The Chords

Because of all the rules associated with four-part harmony writing, sometimes the voicing of a progression leads to a dead-end where the progression is correct, but because of the way a chord is voiced, it may not be able to correctly move to the next chord. This is why for the computer to be able to voice a progression, it must have available all the possible combinations of four note chords. The combinations have restrictions though, for each voice the note must be within that voice’s range, and may not overlap that of another voice.

Checking the Motion Between Chords

To check the motion between chords, the motion itself becomes a Motion object and contains methods to compare the motion with other Motion objects, thus being able to check for all the rules of motion. A Motion object is created with the same voice in two adjacent chords, for all four voices. The object basically takes the two notes, calculates the distance between the two notes, and determined the direction that they are moving. Besides the Motion object, which checks for invalid parallel motion, the program also checks for cross-voicing within chords, cross-voicing between chords, and invalid interval skips. Within the same method which checks for all the previous mentioned motions, there are a series of nested Boolean statements which check for proper resolutions of chords based on chord symbols. Each chord is unique in its resolution, with the exception of when a member of a chord has to resolve in the same direction and distance as another. In this case a method was developed to be used for checking the resolution on chords that resolve using only step-wise motion. Otherwise if there is resolution, it directly involves the leading tone or has a completely unique resolution, and must be dealt with individually (Fr+6 and N6 chords).

Choosing the Most Correct Chord

After all the chords have been created and checked for doubling and motion, the chords that remain are all correct, but there is always a most optimal chord to choose. This is done by first looking at the inversion of the chord because chords are generally preferred to be in root position, with the exception of chords which cannot exist in root position. If there is more than one chord in root position, the chord which has the least motion to its already created adjacent chord is chosen. In the case where a chord does not exist in root position, the chord with the least motion to its already created adjacent chord is chosen.