Thursday, October 25, 2007

Flex-Based Message Board

A while ago I finally got fed up with a hosting service I was using for a particular product's web registration service, so I moved everything to a new host and decided to make some needed changes. One change in particular was the message board. I was using phpbb and was spending too much time battling spammers. I originally had it to where anyone could post, because most people with legit questions prefer not to have to create accounts. This of course was fine until the spammers found it, so I turned on the all the validation stuff and people just stopped using the board entirely (and emailing me questions). With the new host I was in need of a new message board, so I decided to make one myself using Flex (and PHP5). This is the current board: http://www.vdrillpro.net/flexboard/

My concept is to allow people to post semi-anonymously by allowing users to use any name they want, and all they have to do it leave a valid email address that no one can see (but admins). Users then have to enter a validation code, and are then sent an email in which they have to click on the provided link to move their message out of the queue and on to the board to be seen by others. I am hoping that the anonymous posting will encourage more legit posters, and that with the email validation it will slow down the spam but not be too much of a pain. I also built in the ability to ban email addresses by regular expression, which allows me to use keywords and do stuff like ban all .info, .biz, .cn, and .ru addresses.

The layout is also different, displaying topics and comments in grid form to give it more of a desktop application feel:

Topic view allows the user to see the full content of the post, and all of the replies in grid form:

 

Replies also allow rich text with links:

An validation code must also be entered before a post is queued:

 

An email is then sent to the address that was entered containing a link. When the user clicks on that link the post is out of the queue and can then be seen on the message board.

Sunday, October 21, 2007

Introduction to the Spring Framework

What is the Spring Framework?

According to SpringFramework.org:

Spring provides a light-weight solution for building enterprise-ready applications, while still supporting the possibility of using declarative transaction management, remote access to your logic using RMI or web services, and various options for persisting your data to a database [1].

More specifically Spring is a way of using regular Java Beans as Enterprise JavaBeans without involving the complexity of J2EE. JavaBeans encapsulate many objects into a single object (the bean), so that the bean can be passed around rather than the individual objects [2]. The Enterprise JavaBean (EJB) is a managed, server-side component architecture for modular construction of enterprise applications [3], which requires a J2EE container/application server. Java 2 Enterprise Edition (J2EE) is a combination of libraries and useful technologies for the Java programming language, that are used to develop and deploy enterprise applications using modular components using a Java application server.

Spring is more than just JavaBeans though, it is a framework that contains other frameworks, which are designed to independently work well with other frameworks. These frameworks or modules are typically divided into the following:

  • Inversion of Control Container (Core Container)
  • Context Module
  • Aspect Oriented Programming (AOP) Module
  • Data Access Object (DAO) Module
  • Object-Relational Mapping (ORM) Module
  • Web Module
  • Model-View-Controller (MVC) Framework

For the purposes of introduction though, it is best to start with the basics of the Inversion Control Container. Central in the Spring Framework is its Inversion of Control Container that provides a consistent means of configuring and managing Java objects. This container is also known as BeanFactory, ApplicationContext or Core container [4]. Inversion of Control is a concept in which the control flow is inverted compared to the traditional interaction model expressed in imperative style by a series of procedure calls. Thus, instead of the programmer specifying, by the means of function calls, a series of events to happen during the lifetime of a programme, they would rather register desired responses to particular happenings, and then let some external entities take over the control over the precise order and set of events to happen [5].

Where do I get the Spring Framework and how do I install it?

The framework is a series of JAR files, which can be downloaded from http://www.springframework.org/download for fee. To install it all you really have to do is extract the downloaded compressed file to some location on your computer, and then reference the appropriate JARs in a Java project. If you download the full distribution you will get the following directories, which are specified "RELEASE INFO" section of the "readme.txt" included with the distribution:

  • dist -  contains the Spring distribution jar files, as well as a zip of all Java source files
  • docs -  contains the Spring reference documentation in PDF and HTML format, as well as the complete API javadocs
  • lib -  contains all third-party libraries needed for building the framework and/or running the samples
  • src -  contains the general Java source files for the framework
  • mock -  contains the general Java source files for Spring's mock and test classes
  • test -  contains the general Java source files for Spring's test suite
  • tiger/src - contains the JDK-1.5-specific Java source files for the framework
  • tiger/test -  contains the JDK-1.5-specific Java source files for Spring's test suite
  • aspectj/src -  contains the AspectJ-specific source files for the framework
  • aspectj/test - contains the AspectJ-specific source files for Spring's test suite
  • samples - contains various demo applications and showcases

How do I use the Spring Framework?

Considering that there are many sub-frameworks in Spring, it is easiest to start with an example that demonstrates the core basics of the Inversion of Control Container. There are many available Spring Tutorials that can be found in books and on the internet. So far the best example that I have been able to find is the Inversion of Control example from DevelopersBook.com, in which you don't directly connect your components and services together in code but describe which services are needed by which components in a configuration file [6]. I have taken the basic example and modified it to include comments and a few other properties, with the intention for later use in an Aspect Oriented Programming example. Consider a simple application framework that provides JavaBean that is used to greet as specified person using a "Greeting Service," and say good-bye to that person using a "Leaving Service." This requires the following classes, interfaces, and XML configuration:

  • HelloBean.java - This is the bean that is used to encapsulate the "Greeting Service" and "Leaving Service."
  • HelloBean.xml - This is the Spring XML configuration file that is used to specify the instance and relationships of the involved object.
  • GreetingService.java (implementing GreetingInterface.java) - This is the service that is used to generate the greeting message.
  • LeavingService.java (implementing LeavingInterface.java) - This is the service that is used to generate the leaving message.
  • HelloClient.java - This is the main class that uses the Inversion of Control Container with the Spring XML configuration file to get the specified instance of the HelloBean.

The following is the project directory structure, which shows the location of the Java source code files, the related class files, and the Spring XML configuration file:

  • bin
    • springapp
      • service
        • interfaces
          • GreetingInterface.class
          • LeavingInterface.class
        • GreetingService.class
        • LeavingService.class
      • bean
        • HelloBean.class
      • HelloClient.class
    • HelloBean.xml
  • src
    • springapp
      • service
        • interfaces
          • GreetingInterface.java
          • LeavingInterface.java
        • GreetingService.java
        • LeavingService.java
      • bean
        • HelloBean.java
      • HelloClient.java

Architecturally the following class diagram shows how all of the objects relate to one another within the "springapp" package:

The "HelloClient" uses the Spring XML configuration file "HelloBean.xml" with  the Inversion of Control Container to create the specified instance of the "HelloBean." The "HelloBean" encapsulates the objects and functionality of the "GreetingService" and "LeavingService."

For this particular application, the following libraries from the Spring Framework are required:

  • <spring-install>\lib\jakarta-commons\*.jar
  • <spring-install>\lib\log4j\lg4j-1.2.14.jar
  • <spring-install>\dist\spring.jar

GreetingService.java

This is an object that is instantiated by the "HelloBean" in order to generate the greeting message. The actual instantiation though is specified in the "HelloBean.xml" file, which is handled by the Inversion of Control Container or BeanFactory. This class only has one property, which is the variable "greetingMessage." In order for the BeanFactory to be able to allow the setting of the value of this variable the class must provide a "setter" method. In the case of the "GreetingService," it has a "setGreetingMessage" method that can used by the BeanFactory to set the value of the "greetingMessage" variable.

package springapp.service;

import springapp.service.interfaces.GreetingInterface;

/**
 * <p>Title: Greeting Service</p>
 * 
 * <p>Description: This is the service that is used to generate
 * the greeting message.</p>
 * <p>Taken and modified from Code Listing 1:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class GreetingService implements GreetingInterface {

 /** This is greeting message that is displayed.*/
 private String greetingMessage;

 /**
  * Returns the "Hello" message for the person of the given name.
  * @param name
  * @return String
  */
 public String sayHello(String name) {
  return greetingMessage + " " + name + ".";
 }

 /**
  * Sets the greeting message
  * @param message
  */
 public void setGreetingMessage(String message) {
  this.greetingMessage = message;
 }

}

GreetingInterface.java

The methods required to be implemented by the "GreetingService" that are public and are not "getter" or "setter" methods are specified in this interface. This is because it is considered good practice to separately specify those methods in an implemented interface.

package springapp.service.interfaces;

/**
 * <p>Title: Greeting Interface</p>
 * 
 * <p>Description: Representing the methods of the Greeting Service.</p>
 * <p>Taken and modified from Code Listing 1:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public interface GreetingInterface {

 /**
  * Returns the "Hello" message for the person of the given name.
  * @param name
  * @return String
  */
 public String sayHello(String name);
}

LeavingService.java

This is an object that is instantiated by the "HelloBean" in order to generate the leaving message. The actual instantiation though is specified in the "HelloBean.xml" file, which is handled by the Inversion of Control Container or BeanFactory. This class only has one property, which is the variable "leavingMessage." In order for the BeanFactory to be able to allow the setting of the value of this variable the class must provide a "setter" method. In the case of the "LeavingService," it has a "setLeavingMessage" method that can used by the BeanFactory to set the value of the "leavingMessage" variable.

package springapp.service;

import springapp.service.interfaces.LeavingInterface;

/**
 * <p>Title: Leaving Service</p>
 * 
 * <p>Description: This is the service that is used to generate
 * the leaving message.</p>
 * <p>Taken and modified from Code Listing 2:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class LeavingService implements LeavingInterface {

 /** The good-bye message to be displayed. */
 private String leavingMessage;

 /**
  * Returns the "Good-Bye" message for the person of the given name
  * @param name
  * @return String
  */
 public String sayGoodBye(String name) {
  return leavingMessage + " " + name + ".";
 }

 /**
  * Sets the good-bye message
  * @param message
  */
 public void setLeavingMessage(String message) {
  this.leavingMessage = message;
 }

}

LeavingInterface.java

The methods required to be implemented by the "LeavingService" that are public and are not "getter" or "setter" methods are specified in this interface.

package springapp.service.interfaces;

/**
 * <p>Title: Leaving Interface</p>
 * 
 * <p>Required methods for the Leaving Service.</p>
 * <p>Taken and modified from Code Listing 2:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 * 
 * @author John Valentino II
 *
 */
public interface LeavingInterface {

 /**
  * Returns the "Good-Bye" message for the person of the given name
  * @param name
  * @return String
  */
 public String sayGoodBye(String name);

}

HelloBean.java

This is the JavaBean that is used to encapsulate the services of this example application. Since this class has three properties, it specifies three "setter" methods to allow the setting of instances or values by the BeanFactory or other parent classes.

package springapp.bean;

import springapp.service.GreetingService;
import springapp.service.LeavingService;

/**
 * <p>Title: Hello Bean</p>
 * 
 * <p>Description: Represents the Hello Bean.</p>
 * <p>Taken and modified from Code Listing 4:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class HelloBean {

 /** Represents the service used to generate a greeting */
 private GreetingService greetingService;
 /** Represents the service used to generate a good-bye message */
 private LeavingService leavingService;
 /** Represents the name of the person to meet or leave */
 private String personName;

 /**
  * Sets the name of the person
  * @param name
  */
 public void setPersonName(String name) {
  this.personName = name;
 }

 /**
  * Sets the instance of the greeting service
  * @param service
  */
 public void setGreetingService(GreetingService service) {
  this.greetingService = service;
 }

 /**
  * Set the instance of the leaving service
  * @param service
  */
 public void setLeavingService(LeavingService service) {
  this.leavingService = service;
 }

 /**
  * Generates the greeting message
  * @return
  */
 public String greet() {
  return greetingService.sayHello(personName);
 }

 /**
  * Generates the leaving message
  * @return
  */
 public String leave() {
  return leavingService.sayGoodBye(personName);
 }

}

HelloClient.java

This is the main class of the application that uses the "HelloBean.xml" with the Inversion of Control Container to create the instance of the "HelloBean" and call is available methods.

package springapp;

import springapp.bean.HelloBean;
import org.springframework.context.support.ClassPathXmlApplicationContext;

/**
 * <p>Title: Hello Client</p>
 * 
 * <p>Description: Represents a program that uses the HelloBean.</p>
 * <p>Taken and modified from Code Listing 5:
 * http://www.developersbook.com/spring/spring-tutorials/</p>
 *
 * @author John Valentino II
 */
public class HelloClient {

 /**
  * @param args
  */
 public static void main(String[] args) {

  //Get the instance of the HelloBean as specified in HelloBean.xml
  ClassPathXmlApplicationContext appContext = 
   new ClassPathXmlApplicationContext(new String[] { "HelloBean.xml" });
  HelloBean helloBean = (HelloBean) appContext.getBean("helloBean");

  System.out.println(helloBean.greet());
  System.out.println(helloBean.leave());

 }

}

HelloBean.xml

This is the Spring XML configuration file that is used to specify the instances and properties of the bean classes within the application.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" 
"http://www.springframework.org/dtd/spring-beans.dtd">

<beans>
 <!-- Creates an instance of the "HelloBean" class -->
 <bean id="helloBean" class="springapp.bean.HelloBean">
  <!-- Sets the value of "personName" by
       calling "setPersonName" -->
  <property name="personName">
   <value>Sarah</value>
  </property>
  <!-- Sets the instance of "greetingService" by
       calling "setGreetingService" -->
  <property name="greetingService"> 
   <ref bean="greetingService" />
  </property>
  <!-- Sets the instance of "leavingService" by
       calling "setLeavingService" -->
  <property name="leavingService">
   <ref bean="leavingService" /> 
  </property> 
 </bean>
 <!-- Creates an instance of the "GreetingService" -->
 <bean id="greetingService" class="springapp.service.GreetingService">
  <!-- Sets the value of "greetingMessage" by 
         calling "setGreetingMessage" -->
  <property name="greetingMessage">
   <value>Hello</value>
  </property>
 </bean>
 <!-- Creates an instance of the "LeavingService" -->
 <bean id="leavingService" class="springapp.service.LeavingService">
  <!-- Sets the value of "leavingMessage" by
       calling "setLeavingMessage" -->
  <property name="leavingMessage">
   <value>Good-bye</value>
  </property>
 </bean>
</beans>

Output

The program runs and generates the following output:

Hello Sarah.
Good-bye Sarah.

This output is to be expected as the Spring XML configuration file specifies that the "HelloBean" instance has the specified "personName," an instance of the "GreetingService," and an instance of the "LeavingService." Both service instances are also given specified "greetingMessage" and "leavingMessage" values.

Links

References

  1. SpringFramework.org, Documentation Preface, http://static.springframework.org/spring/docs/2.0.x/reference/preface.html
  2. Wikipedia, JavaBean, http://en.wikipedia.org/wiki/JavaBeans
  3. Wikipedia, Enterprise JavaBean, http://en.wikipedia.org/wiki/Ejb
  4. Wikipedia, Spring Framework, http://en.wikipedia.org/wiki/Spring_framework
  5. Wikipedia, Inversion of Control, http://en.wikipedia.org/wiki/Inversion_of_Control
  6. DevlopersBook.com, Spring Framework Tutorial, http://www.developersbook.com/spring/spring-tutorials/spring-tutorials.php

Friday, May 11, 2007

Instant Bach goes Open Source

Introduction

The program "Instant Bach" was made as a project for an unnamed computer science class, which was when I was a sophomore computer science major, and a senior music performance and music composition double-major. Needless to say at the time of this program's creation I knew far more about musical theory than I did computer science. When I approached the professor of this class with my project idea his response was that "other people have tried stuff similar for this class and failed, and the same will happen to you." Despite this opinion I remained confident that a four-part harmony writing program could be made using the skills I currently had available, so I went ahead and made the program for my project. I have recently decided to resurrect this program to expand its capabilities as an open source project (http://sourceforge.net/projects/instantbach).

The original program was based of of static rules which limits it to a particular number of possible progressions:

My most immediate goal for this project is to move all of the rules of voice leading, doubling, chord structure, and progression to an external rules file, so that the program can be adjusted to work beyond just being able to voice major key progressions.

Purpose

The first two years of undergraduate music theory are spent learning and doing mostly one thing: four-part harmony writing. The purpose of this program is to be able to do four part harmony writing, with or without a user specified progression, according to all the rules of four part harmony writing, and have the ability to transpose the result into any key.

Musical Intervals

The distance between any two musical notes is referred to as an interval, and is represented by a two-character symbol. The first character denotes the quality, being P for perfect, A for augmented, D for diminished, m for minor, and M for major. The second character denotes the melodic distance with a number of how many lines and spaces exist between the two notes of the interval. Major and minor refer to whether an interval occurs within a natural minor scale or within a major scale. Augmented and diminished are used for intervals that do not exist in the major or natural minor scales, augmented being greater than, and diminished being less than. The following is a chart of the most common musical intervals:

Tertian Harmony

Tertian Harmony refers to simultaneously sounding pitches in which each of the pitches are separated by the distance of major or minor third, in at least one permutation of the order of notes. Tertian harmonies consisting of three distinct notes are referred to as triads, and harmonies consisting of four distinct notes are referred to as seventh chords. A triad can have four different classifications depending on the interval relationship between the notes in the triad. These different types of triads are called major, minor, augmented, and diminished. Seventh chords can be major-major, dominant, minor, half-diminished, or fully diminished.

Progression

For any given scale, for example a C Major scale (C, D, E, F, G, A, B), each of the pitches in that scale represent the root note of a tertian chord of some type. Each of these harmonies are referred to as chords and are represented by a roman numeral. The roman numeral indicates the quality of a chord (major, minor, augmented, diminished), the position of the chord inside the key, and which note is on the bottom, (the root), of that chord.

The first note of a chord is known as the root, the second is the third, the third is the fifth, and if there is a fourth note it is called the seventh. Whenever a chord cannot be spelled in thirds, it is in an inversion, meaning that a note other then the root is on the bottom of the chord. An inversion is denoted by one number on top of another and is written to the right of the chord symbol, but I will write them as fractions for simplicity. In a triad, when the third is on the bottom of the chord it is in first inversion and the symbol that is used is 6. When the fifth is on the bottom of a triad it is in second inversion and is written as 6/4. Seventh chords can have three inversions, with the third, fifth, and seventh on the bottom, denoted 6/5, 4/3, and 2.

Rules of Progression

The order in which chords can occur have multiple rules, the easiest rule being that I can move to anything, but rather than explain the progression behavior with words, I will use a picture of the progression graph, where each chord symbol is a vertex, and all edges are un-weighted.

Other Chord Symbols

As shown in the graph, there are other chord symbols that have not yet been described. They are the Neopolitan Sixth Chord (N6), the French Augmented Sixth Chord (Fr+6), and the secondary dominant (V/x). These chords incorporate notes that are outside of the given key, and have strict rules for which chord they can lead to. The N6 and Fr+6 must go to V, and secondary dominants must lead to chord symbol on the bottom of the fraction. Secondary dominants are considered to be a borrowed V from another key, and therefore must progress to the I of the key from which they were borrowed.

Four part Harmony Writing

Four-part harmony writing is when a progression is developed and the notes in the chords of that progression are assigned to four voices, the bass, tenor, alto, and soprano. Each voice has to stay within a particular range and is limited to the distance which the voice can move between chords. Each note has rules for motion and each chord has rules for its resolution. Because most of the progression consists of triads, and there are four voices, a member of the chord has to be doubled, which there are also rules for.

Rules of Doubling

  • I, ii, IV, iii, and vi : The root and third can be doubled.
  • V and V/x : The root can be doubled.
  • V7 and V7/x : no doubling is needed or the root can be doubled and the fifth omitted.
  • viio : The third can be doubled.
  • vii07 : No doubling is needed.
  • N6 : The third can be doubled.
  • Fr+6 : No doubling is needed.

Rules of Resolution

  • I, ii, IV, and vi : None.
  • iii : If the fifth is in the soprano or bass it must resolve up by a m2. If the fifth is in the tenor or alto it must either resolve up a m2 or move down a M3.
  • V and V/x : If the third is in the soprano or bass it must resolve up by a m2. If the third is in the tenor or alto it must either resolve up a m2 or move down a M3.
  • V7 and V7/x : The third must resolve as the same as in the V and V/x and the seventh must move down by a m2.
  • viio : If the root is in the soprano or bass it must resolve up by a m2. If the root is in the tenor or alto it must either resolve up a m2 or move down a M3. If the chord which the viio is going to is a V, then the leading tone (the seventh degree of the scale) is already present and there are no rules of resolution.
  • vii07 : The root must resolve the same as in the viio, and even if the next chord is V, the seventh must move down by a M2.
  • N6 : The root must move down by a D3.
  • Fr+6 : The A6 (or its inversion d3) interval in the chord must resolve to P8 or P1.

Rules of Individual Voice Motion

  • Bass: The bass generally skips around more than all of the other voices and does not have any specific restrictions.
  • Tenor and Alto: Both these voices generally move very little and rarely move more then the interval of a third.
  • Soprano: This voice moves more then the tenor and alto but is usually limited to skips of less then a P4.

Range of Voices in Relation to Each Other

The distance between the tenor and the bass can never exceed a P8+ M3, while the distance between all other voices must remain within a P8.

Cross-Voicing

All voices must always be in order from lowest to highest: bass, tenor, alto, and soprano. If a voice is ever out of that order it is considered cross-voicing, which is not allowed. Cross-voicing can also occur in the motion between two chords.

Illegal Parallel Motion

If two voices move the same interval in the same direction they are in parallel motion. The parallel motions of a P8, P5, or P1 are illegal parallel motions. These are called parallel octaves, fifths, and unisons.

Programming the Progression

The progression for the program can be created two ways, either the user can select the individual chords, or the computer can generate the progression itself. Harmonic progression can be represented as an unweighted graph, in order to determine whether a specified progression is valid. The main problem with this approach is that if the computer were to generate its own progression, it would have to end at a specific point with a cadence. (A cadence is a series of chords which end a musical idea or statement). The computer is able to do two of the most common cadences, an imperfect authentic cadence (IAC), and a perfect authentic cadence (PAC). Each of the cadences consists of a V then I, but in the PAC the soprano shares the root with the bass. The problem is that if you have n chords and start with a random chord, and randomly move through the graph until n-2, then cadence, there is a good chance that the chord at n-3 will not be able to go to the chord at n-2. This would mean that the progression would have to be re-done. To prevent this problem, I built the graph backwards, and the progression is formed backwards as well. This way the progression never runs itself into a dead-end because it always starts with I, V, and then the random chords, so when the progression reaches n, the chord at n will always exist because there are no verticies in the graph that do not have at least one adjacent vertex.

Turning the Progression into Musical Notes

After the progression has been created, the computer needs a way to assign notes of all chords in the progression to the four voices. To assign notes to the four voices the computer must first generate all possible notes each voice can have. This is done by using a recursive method that when given the starting pitch of a chord, and the symbol of that chord, builds a chord consisting of every possible note in that chord. Because rules of voice leading depend on individual chords and the position of individual notes within that chord, each note in a chord maintains whether a given note is a root, third, fifth, or seventh.

The Chord as a Data Structure

The main object of this program is the chord, which consists of four voices and a chord symbol, so the obvious main data structure for the program is a Chord object, consisting of four voices and a chord symbol. A Chord object also contains methods which allow the individual voices to be viewed as either the note or the position of that note within the chord, as well as a method to obtain the chord symbol. This allows other methods to deal with notes arithmetically and as the position of that note within the chord for voice leading purposes.

Creating All The Chords

Because of all the rules associated with four-part harmony writing, sometimes the voicing of a progression leads to a dead-end where the progression is correct, but because of the way a chord is voiced, it may not be able to correctly move to the next chord. This is why for the computer to be able to voice a progression, it must have available all the possible combinations of four note chords. The combinations have restrictions though, for each voice the note must be within that voice’s range, and may not overlap that of another voice.

Checking the Motion Between Chords

To check the motion between chords, the motion itself becomes a Motion object and contains methods to compare the motion with other Motion objects, thus being able to check for all the rules of motion. A Motion object is created with the same voice in two adjacent chords, for all four voices. The object basically takes the two notes, calculates the distance between the two notes, and determined the direction that they are moving. Besides the Motion object, which checks for invalid parallel motion, the program also checks for cross-voicing within chords, cross-voicing between chords, and invalid interval skips. Within the same method which checks for all the previous mentioned motions, there are a series of nested Boolean statements which check for proper resolutions of chords based on chord symbols. Each chord is unique in its resolution, with the exception of when a member of a chord has to resolve in the same direction and distance as another. In this case a method was developed to be used for checking the resolution on chords that resolve using only step-wise motion. Otherwise if there is resolution, it directly involves the leading tone or has a completely unique resolution, and must be dealt with individually (Fr+6 and N6 chords).

Choosing the Most Correct Chord

After all the chords have been created and checked for doubling and motion, the chords that remain are all correct, but there is always a most optimal chord to choose. This is done by first looking at the inversion of the chord because chords are generally preferred to be in root position, with the exception of chords which cannot exist in root position. If there is more than one chord in root position, the chord which has the least motion to its already created adjacent chord is chosen. In the case where a chord does not exist in root position, the chord with the least motion to its already created adjacent chord is chosen.

Sunday, April 29, 2007

Creating a Source Code Syntax Parser in Java Using ANTLR

Recently I found myself looking for a way to parse through the source code syntax of several languages. Rather than write my own syntax parser, I began looking for an existing open source solution. The first option I came across was JavaCC (https://javacc.dev.java.net/), which has a lot of documentation regarding how to install it, create grammars, create parsing code for grammars, but not really much on how to use it. Also according to the JavaCC FAQ:

JavaCC does not automate the building of trees (or any other specific parser output), although there are at least two tree building tools JJTree and JTB (see Chapter 6.) based on JavaCC, and building trees "by hand" with a JavaCC based parser is easy. (http://www.engr.mun.ca/~theo/JavaCC-FAQ/javacc-faq-moz.htm)

While searching for an easy way to for building trees "by hand" using JavaCC I came across another tool called ANTLR. ANTLR, ANother Tool for Language Recognition, (formerly PCCTS) is a language tool that provides a framework for constructing recognizers, compilers, and translators from grammatical descriptions containing Java, C#, Python, or C++ actions (http://www.antlr.org/about.html). With an abundance of grammars (http://www.antlr.org/grammar/list) and articles (http://www.antlr.org/article/list) it was pretty easy to get what I wanted working fast.

My specific example is a program that parses Java 1.5 source code for is imports, its "has a" relationships, its "is a" relationships, and the classes that it realizes (implements). Here are the steps to getting it to work:

  1. Download and install ANTLR (version 2.7.7) from http://www.antlr.org/download.html
  2. Download the Java 1.5 grammar for ANTLR 2.7.7 by Michael Studman's located at http://www.antlr.org/grammar/1090713067533/index.html.
  3. If running on Windows, add the location of ANTLR to the CLASSPATH environment variable. For example my library is located in "C:\antlr\277\lib\antlr.jar".
  4. Open a command window and navigate to the location of the grammar file, for example I put the Java 1.5 grammar in "C:\antlr\277\examples\java15-grammar".
  5. Run the command "java antlr.Tool java15.tree.g java15.g" assuming you are using Michael Studman's Java 1.5 grammar. This will generate the source code for the parser to be included in your project:
    • JavaLexer.java
    • JavaRecognizer.java
    • JavaTokenTypes.java
    • JavaTreeParser.java
    • JavaTreeParserTokenTypes.java
  6. Create a Java project using the above files along with the ANTLR JAR, and then use the following as the main class:
package antlrtest;

// Standard I/O
import java.io.IOException;
import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
// ANTLR Library
import antlr.collections.AST;
import antlr.debug.misc.ASTFrame;
import antlr.RecognitionException;
import antlr.TokenStreamException;
import antlr.CommonAST;
import antlr.ASTFactory;
// Events
import java.awt.event.WindowAdapter;
import java.awt.event.WindowEvent;
// Data
import java.util.ArrayList;

/**
 * <p>Title: Main</p>
 *
 * <p>Description: Takes a Java source code file as an argument, parses it,
 * displays it content in a tree, and determines its imports and relationships
 * with other classes.</p>
 *
 * <p>Copyright: Copyright (c) 2007</p>
 *
 * @author John Valentino II
 * @version 1.0
 */
public class Main implements JavaTokenTypes {

    /**
     * Entry points for the program
     * @param args String[]
     */
    public static void main(String[] args) {

        if (args.length != 1) {
            System.out.println("Usage: main <java file name>");
            System.exit(0);
        }

        try {

            ArrayList<String> imports = new ArrayList<String>();
            ArrayList<String> is_a = new ArrayList<String>();
            ArrayList<String> realizes = new ArrayList<String>();
            ArrayList<String> has_a = new ArrayList<String>();

            //Open the given file
            File file = new File(args[0]);
            String fileName = file.getName();
            BufferedReader reader = new BufferedReader(new FileReader(file));

            //Create a scanner that reads from the input stream passed to us
            JavaLexer lexer = new JavaLexer(reader);
            lexer.setFilename(fileName);

            //Create a parser that reads from the scanner
            JavaRecognizer parser = new JavaRecognizer(lexer);
            parser.setFilename(fileName);

            //start parsing at the compilationUnit rule
            parser.compilationUnit();

            //Get the imports and relationships for this Java file and returns
            //the full class name
            String fullClassName = parseTree(parser.getAST(),
                parser.getTokenNames(),
                imports, is_a, realizes, has_a);

            System.out.println("Full Class Name: " + fullClassName);

            //display imports
            System.out.println("Imports:");
            for (int i = 0; i < imports.size(); i++) {
                System.out.println("\t" + imports.get(i));
            }

            //display the classes that this class inherits from
            System.out.println("Extends (Is A):");
            for (int i = 0; i < is_a.size(); i++) {
                System.out.println("\t" + is_a.get(i));
            }

            //display the "has a" relationships of this class
            System.out.println("Has A: ");
            for (int i = 0; i < has_a.size(); i++) {
                System.out.println("\t" + has_a.get(i));
            }

        } catch (IOException e) {
            e.printStackTrace();
        } catch (RecognitionException re) {
            re.printStackTrace();
        } catch (TokenStreamException tse) {
            tse.printStackTrace();
        }
    }


    /**
     * Parses the AST Root Node for a Java class. The AST Root node contains
     * the following nodes: PACKAGE_DEF, IMPORT, CLASS_DEF
     * @param t AST
     * @param tokenNames String[]
     * @param imports ArrayList
     * @param is_a ArrayList
     * @param realizes ArrayList
     * @param has_a ArrayList
     * @return String
     */
    private static String parseTree(AST t, String[] tokenNames,
        ArrayList<String> imports,
        ArrayList<String> is_a,
        ArrayList<String> realizes,
        ArrayList<String> has_a) {

        String packageName = "";
        String className = "";

        ((CommonAST) t).setVerboseStringConversion(true, tokenNames);
        ASTFactory factory = new ASTFactory();
        AST r = factory.create(0, "AST ROOT");
        r.setFirstChild(t);

        displayFrame(r);

        AST c = r.getFirstChild();

        //for each child...
        while (c != null) {

            int type = c.getType();

            switch (type) {

            case PACKAGE_DEF:
                packageName = getFormattedImport(c);
                break;
            case IMPORT:
                //get the import in the format "java.io.File"
                String importText = getFormattedImport(c);
                //add this import
                imports.add(importText);
                break;
            case CLASS_DEF:
                //there should only be one class per file
                className = parseClass(c, is_a, realizes, has_a);
                break;
            }

            c = c.getNextSibling();

        } //end for each child

        return packageName + "." + className;

    }

    /**
     * Displays the tree in a frame
     * @param r AST
     */
    private static void displayFrame(AST r) {
        final ASTFrame frame = new ASTFrame("Java AST", r);
        frame.setVisible(true);
        frame.addWindowListener(new WindowAdapter() {
            public void windowClosing(WindowEvent e) {
                frame.setVisible(false);
                frame.dispose();
                System.exit(0);
            }
        });
    }

    /**
     * Parses an IMPORT node and returns the full name
     * of the package
     * @param root AST
     * @return String
     */
    private static String getFormattedImport(AST root) {

        //get the import in the format "java.io.File."
        String importText = parseImport(root);

        //trim off the last character making it "java.io.File"
        importText = importText.substring(0, importText.length() - 1);

        return importText;

    }

    /**
     * Recursively parses an IMPORT node for the full name of the package
     * @param root AST
     * @return String
     */
    private static String parseImport(AST root) {

        String returner = "";

        AST c = root.getFirstChild();

        //for each child...
        while (c != null) {
            int type = c.getType();
            if (type == DOT)
                returner = parseImport(c);
            else {
                if (!c.getText().equals(""))
                    returner += c.getText() + ".";
            }
            c = c.getNextSibling();
        } //end for each child

        return returner;

    }

    /**
     * Parses the given CLASS_DEF node, which contains the following nodes:
     * MODIFIERS, EXTENDS_CLAUSE, IMPLEMENTS_CLAUSE, OBJBLOCK, IDENT
     * @param root AST
     * @param is_a ArrayList
     * @param realizes ArrayList
     * @param has_a ArrayList
     * @return String
     */
    private static String parseClass(AST root,
        ArrayList<String> is_a,
        ArrayList<String> realizes,
        ArrayList<String> has_a) {

        String className = "";

        AST c = root.getFirstChild();

        //for each child...
        while (c != null) {
            int type = c.getType();

            switch (type) {
            case IDENT:
                className = c.getText();
                break;
            case EXTENDS_CLAUSE:
                parseImplementsOrExtensions(c, is_a);
                break;
            case IMPLEMENTS_CLAUSE:
                parseImplementsOrExtensions(c, realizes);
                break;
            case OBJBLOCK:
                parseObject(c, has_a);
                break;
            }

            c = c.getNextSibling();

        } //end for each child

        return className;

    }

    /**
     * Parses the given IMPLEMENTS_CLAUSE or EXTENDS_CLAUSE to determine the
     * classes that are implements or extended
     * @param root AST
     * @param realizes ArrayList
     */
    private static void parseImplementsOrExtensions(AST root,
        ArrayList<String> realizes) {

        AST c = root.getFirstChild();

        //for each child...
        while (c != null) {

            int type = c.getType();

            switch (type) {
            case IDENT:
                realizes.add(c.getText());
                break;
            }

            c = c.getNextSibling();

        } //end for each child

    }

    /**
     * Parses the given OBJBLOCK node, which contains the following nodes:
     * VARIABLE_DEF, METHOD_DEF, CLASS_DEF, CTOR_DEF, IDENT
     * @param root AST
     * @param has_a ArrayList
     */
    private static void parseObject(AST root, ArrayList<String> has_a) {

        AST c = root.getFirstChild();

        //for each child...
        while (c != null) {
            int type = c.getType();

            switch (type) {
            case VARIABLE_DEF:
                parseVariableDef(c, has_a);
                break;
            }

            c = c.getNextSibling();

        } //end for each child

    }

    /**
     * Parses the given VARIABLE_DEF node, which contains the following nodes:
     * MODIFIERS, TYPE, IDENT
     * @param root AST
     * @param has_a ArrayList
     */
    private static void parseVariableDef(AST root, ArrayList<String> has_a) {
        AST c = root.getFirstChild();

        //for each child...
        while (c != null) {

            int type = c.getType();

            switch (type) {
            case TYPE:
                parseType(c, has_a);
                break;
            }
            c = c.getNextSibling();

        } //end for each child

    }

    /**
     * Parses the given TYPE node for its class names
     * @param root AST
     * @param has_a ArrayList
     */
    private static void parseType(AST root, ArrayList<String> has_a) {
        AST c = root.getFirstChild();

        //for each child...
        while (c != null) {

            int type = c.getType();

            switch (type) {
            case IDENT:
                has_a.add(c.getText());
                break;   
            }
            c = c.getNextSibling();

        } //end for each child

    }

}

Using the following the test file UMLHelper.java as the input to the above program:

package a.b.c.umlhelper;

import umlhelper.service.ServiceInterface;
import umlhelper.gui.MainFrameInterface;
import umlhelper.gui.MainFrame;
import umlhelper.service.Service;
import java.io.File;
import a.b.c.d.E;

/**
 * <p>Title: UML Helper</p>
 *
 * <p>Description: This is a test class for parsing, it does not compile.</p>
 *
 * <p>Copyright: Copyright (c) 2007</p>
 *
 * @author John valentino II
 * @version 1.0
 */
public class UMLHelper extends AB implements ServiceInterface, MainFrameInterface, A, B, C {

    /** Represents the gui tier of the application */
    private MainFrame mainFrame;
    /** Represents the service tier of the application */
    private Service service;
    /** Represents the version of this program */
    private float version = 0.1f;
    private ArrayList<String> list;

    /**
     * Entry point for the application
     * @param args String[]
     */
    public static void main(String[] args) {
        new UMLHelper();
    }

    /**
     * Creates the application
     */
    public UMLHelper() {
        this.service = new Service(this);
        this.mainFrame = new MainFrame(this);
    }

    /**
     * Returns the vesion of this program
     * @return float
     */
    public float getVersion() {
        return this.version;
    }

    /**
     * Opens the given directory and searches it for source code files and
     * other directories
     * @param directory File
     */
    public void open(File directory) {
        service.open(directory);
    }


}

The following output is generated:

Full Class Name: a.b.c.umlhelper.UMLHelper
Imports:
    umlhelper.service.ServiceInterface
    umlhelper.gui.MainFrameInterface
    umlhelper.gui.MainFrame
    umlhelper.service.Service
    java.io.File
    a.b.c.d.E
Extends (Is A):
    AB
Has A:
    MainFrame
    Service
    ArrayList

The following tree is displayed:

Monday, April 23, 2007

Introduction to the Web Enterprise

Table of Contents


Web Servers

The term web server is used to refer to both the physical computer responsible for providing web services, and the software running on that computer that provides the web services (http://en.wikipedia.org/wiki/Web_server). The software responsible for providing web services is also referred to as an application server (http://en.wikipedia.org/wiki/Application_server). The hardware for a good web server is expensive, along with the cost of bandwidth depending on the service provider and traffic. For this reason smaller to medium sized websites are run on web hosts, which are web hosting service providers that provide the web server and application server for multiple entities (http://en.wikipedia.org/wiki/Web_host). The application server of the web host determines the server-side scripting languages, databases, and enterprise frameworks that you can use. The following are some of the more popular application servers, for a list of all web servers see http://en.wikipedia.org/wiki/Comparison_of_web_servers:

IIS (Internet Information Systems)

IIS is a Microsoft technology for running web services on a web server. Typically IIS is used to run the server-side scripting language ASP and the .NET framework. It also is commonly used with the database management system (DBMS) MS Access and MS SQL. IIS comes with Windows Server OS, and with Windows XP and MCE.

Apache

Apache originally was a Unix based application server, but has since been ported to Linux, Windows, and OS X. It is free and open source, and is credited with playing a key role in the growth of the Internet. Apache is typically used to run the server-side scripting language PHP, and the database management system MySQL.

Apache Tomcat

Apache Tomcat is a web container that implements Java Server Pages. Since this is Java based it is cross platform compatible, so it can run on any operating system and can use any DBMS that has a driver for Java. It should also be noted that projects such as Jakarta allow Apache Tomcat to run with IIS.

JAS (Java Application Server)

JAS is a platform used for delivering server-side web content using the J2EE framework, which includes JSP, servlets, and EJBs (Enterprise Java Beans). JAS includes its own database and can work with any other DBMS with a driver for Java.


Layers of Web Programming

Most people consider the web a mysterious random conglomeration of applications and content of different types and uses, with no categories or organization. While the web is a random series of static and dynamic content, the languages used to generate an maintain that content can be grouped by function and usages. Instead of just referring to every individual language used on the web, I find it more intuitive to group languages by overall function, and divide functions into layers.

  • Text formatting - These are languages that are just used to specify how to handle text in browsers.
  • Browser Scripting - These are languages that are used to program the behavior of the browser.
  • Server Scripting - These are languages that are used to program the behavior of the server.
  • Databases - This refers to the where data is stored, and the language used to communicate with those databases.
  • Data Formats - This refers to the various formats that are used to pass data on the web, which may or may not be independent of a database.
  • Enterprise Architectures - These are packages of web technologies which encapsulate all of the other layers.

Text Formatting

The text formatting languages on the web are basically variations of the Hyper Text Markup Language (HTML), and Cascading Style Sheets (CSS).

HTML (Hyper Text Markup Language)

HTML is simply a language for formatting text in files with the "htm" or "html " file extension. HTML uses markup tags to define what to do with text when it is interpreted by the browser. For example "<b>This is text</b>" in HTML would be displayed as "This is text" in a browser. The <b> markup tag is used to make text bold. While HTML can be done in any text editor, there are many tools that work like a Word Processor for generating HTML, so that the user doesn't have to remember all of the markup tags.

Tools for generating HTML

HTML Tutorials

CSS (Cascading Style Sheets)

CSS is used to define how to display individual HTML elements, by defining display properties within an HTML page or by linking HTML pages to external files with the "css" file extension.

Browser Scripting

The following are the most common languages and practices for programming the behavior of the browser (list from www.w3schools.com) :

  • JavaScript
  • HTML DOM (Hyper Text Markup Language Document Object Model)
  • DHTML (Dynamic Hyper Text Markup Language)
  • VBScript (Visual Basic Script)
  • AJAX (Asynchronous JavaScript and XML)
  • AppML (Application Markup Language)
  • E4X (ECMAScript for XML)
  • WMLScript (Wireless Markup Language Script)

JavaScript

JavaScript is a scripting language for programming the behavior of the browser, with syntax similar to that of the Java programming language. It should also be noted that JavaScript and the Java programming language are not the same thing. JavaScript can be embedded within an HTML page, or linked externally as a file with a ".js" file extension. Either way JavaScript is executed at runtime by the browser itself in order to provide dynamic interaction.

HTML DOM (Hyper Text Markup Language Document Object Model)

HTML DOM is a standardized set of methods for the management of HTML documents for multiple programming languages, by dividing the HTML document into a node tree:

Image from W3Schools - http://www.w3schools.com/htmldom/dom_nodes.asp

DHTML (Dynamic Hyper Text Markup Language)

DHTML is not a language itself, but is used to describe a combination of web languages used to add dynamic content to HTML. Usually the combination consists of HTML 4.0, CSS, DOM, and JavaScript.

VBScript (Visual Basic Script)

VB Script is a language for programming the behavior of the browser, with syntax similar to Visual Basic. VBScript is executed at runtime by the browser itself in order to provide dynamic interaction.

AJAX (Asynchronous JavaScript and XML)

AJAX is not a language, but is a technique for using JavaScript, XML, HTML, and CSS to create more interactive web pages faster.

AppML (Application Markup Language)

AppML is a declarative language that uses XML to describe internet applications, which is open source and browser independent. AppML XML files reside on a web server and use an AppML web service for execution.

E4X (ECMAScript for XML)

E4X is an standardized extension of JavaScript that includes direct support of XML.

WMLScript (Wireless Markup Language Script)

WMLScript is a browser scripting language for WML pages, which are pages written in an XLM based markup language that inherits from HTML, intended to be viewed by wireless devices.


Server Scripting

Server scripting languages are used to execute code directly on the server. Because these languages are executed on the server, no browser client support is needed. In the 3-tier architecture server side languages reside in the application/logic tier, while in the MVC pattern the server side languages reside in the controller. The following are the most common languages for programming the behavior of the server:

  • ASP (Active Server Pages)

  • ESP (Escapade Server Pages)

  • JSP (JavaServer Pages)

  • CFML (ColdFusion Markup Language)

  • PHP (Hypertext Preprocessor)

ASP (Active Server Pages)

ASP is a Microsoft technology that most commonly runs on the Internet Information Systems (IIS) web service used for generating dynamic web pages. ASP pages can actually be written in either VBScript, PerlScript, or JScript, where VBScript is the most common implementation. Though ASP is still in widespread use it has evolved into ASP.NET as part of the .NET architecture, which is discussed later.

ESP (Escapade Server Pages)

ESP is a server-side scripting language the runs on Linux and Unix based platforms. It is commonly used in Europe but is generally not used within the US.

JSP (JavaServer Pages)

JSP is a Java technology server-side scripting language with XML-like syntax that can run on Tomcat or JAS (Java Application Server), which is part of the J2EE architecture but can be used by itself.

CFML (ColdFusion Markup Language)

CFML is an XML based server-side scripting language that typically runs on the Adobe ColdFusion application server.

PHP (Hypertext Preprocessor)

PHP is a server-side scripting language with syntax similar to PERL. It typically runs on the Apache application server.


Databases

A database is a structured collection of data (http://en.wikipedia.org/wiki/Database), but in web programming the term database usually refers to the software used to manage and query the database. This software is referred to as a Database Management System (DBMS), or more correctly for database systems that are based on the relational model, Relational Database Management Systems (RDBMS). The relational model uses predicate logic and set theory to imply relationships between data within in a database (http://en.wikipedia.org/wiki/Database#Relational_model), which yields the ability to have related data. Communication with databases occurs through the use of queries using the Structured Query Language (SQL), which varies slightly from one database system to the next. Some of the more popular database systems are as follows:

Microsoft Access

Microsoft Access, now called Microsoft Office Access, is a simple relational database management system that what it lacks in power makes up for in simplicity. Microsoft Access doesn't require an application server and can be used as a desktop application, but when run with an application server it is typically IIS.

Microsoft SQL Server

Microsoft SQL Server is a relational database management system that typically runs on IIS.

MySQL

MySQL is a relational database management system that runs on numerous application servers, notably Apache and IIS.

Sybase

Sybase is a relational database management system that at one time was almost identical to MS SQL Server, but has since branched off into its own. It also runs on multiple platforms including Windows (IIS).

Oracle

The term Oracle is used to refer to both a relational database management system and the company that makes it. Oracle runs on multiple platforms, but is the most expensive of RDMS with prices depending on license agreement that range from $149 per user to $40,000 per machine.

DB2

DB2 is IBM's relational database management system, which runs on multiple platforms (Windows, Linux, and Unix).

PostgreSQL

PostgreSQL is an object-relational database management system (ORDBMS), which is a relational database management system that allows developers to integrate the database with their own custom data types and methods (http://en.wikipedia.org/wiki/ORDBMS).


Data Formats

The term data format in the context of this document is used to refer to standardized methods for storing and/or transmitting data with or without the use of a database. All of the formats described here are XML based, which means they use the XML syntax for storing self-describing data.

XML

XML

The Extensible Markup Language (XML) is a general purpose markup language used for storing self-descriptive data in a human readable format using tree-based structures.

XSL

The Extensible Stylesheet Language (XSL) is stylesheet language for XML, and is used to describe how an XML document should be displayed in the same way that CSS does for HTML. XSL consists of XSLT, XSL-FO, and XPath.

XSLT

XSL Transformations (XSLT) is a language for transforming XML documents.

XSL-FO

XSL Formatting Objects (XSL-FO) is a language for formatting XML documents.

XPath

XML Path (XPath) is a language for navigating XML documents.

XQuery

XML Query (XQuery) is a language for querying XML documents similar to SQL.

Linking in XML

XML Link (XLink) defines a format for creating hyperlinks in XML documents, while XML Pointer (XPointer) defines a format for creating hyperlinks to point to more specific parts of XML documents.

DTD

Document Type Definition (DTD) is used to define the structure of XML documents.

XSD

XML Schema Definition (XSD) is an alternative to DTD and is used to define the structure of XML documents.

XML DOM

XML Document Object Model (XML DOM) defines a standard way for traversing and manipulating XML documents.

SOAP

Simple Object Access Protocol (SOAP) is an XML based protocol for communication over HTTP.

WSDL

The Web Services Description Language (WSDL) uses XML to describe web services and how they are accessed.

RDF

The Resource Description Framework (RDF) is a standard for describing web resources.

RSS

Really Simple Syndication is an XML based method distributing web content, most commonly in the form of news feeds.


Enterprise Architectures

In terms of the Web Enterprise, Enterprise Frameworks refer to a group of related services for creating and managing web content over all tiers of web development using defined practices, tools, and methods. Two popular and competing Enterprise Frameworks are .NET and J2EE.

J2EE

Java Platform, Enterprise Edition (Java EE) is the industry standard for developing portable, robust, scalable and secure server-side Java applications (http://java.sun.com/javaee/). As shown in the following figure, J2EE provides the framework for all tiers of web development:

Image from java.sun.com - http://java.sun.com/javaee/5/docs/tutorial/doc/

.NET

The .NET Framework provides a large body of pre-coded solutions to common program requirements, and manages the execution of programs written specifically for the framework (http://en.wikipedia.org/wiki/Microsoft_.NET_Framework#.NET_Framework_architecture). As does J2EE, the .NET Framework encompasses all tiers of web development. It also relies on the Common Language Infrastructure (CLI) to take in code from different languages in order to execute the result on the windows platform:

Image from Wikipedia - http://en.wikipedia.org/wiki/Microsoft_.NET_Framework