Thursday, April 21, 2016

Run Integration Tests Separately Within Your Maven Build

There are several ways to configure Maven to run designated tests separately in a Java project.  Usually, people want to distinguish between unit tests and other types of automated tests during a build.  Unit tests are fast because you are mocking all the external services that the particular code under test is relying upon.  They’re also typically smaller than functional tests, since they are (supposed to be ;) testing a unit of code rather than an entire feature.

However, functional tests are also critical to the success of your project.  You or your managers are probably interested in seeing automated end-to-end usage of your application running constantly without errors, but how is this possible without annoying the developers as they wait for all the tests to finish?

The Maven Failsafe plugin is most helpful in separating unit from functional tests.  By default, it focuses on tests whose filename follows the specific pattern:

**/IT*.java
**/*IT.java
**/*ITCase.java

Of course, you can add (or even exclude) files of particular naming patterns by modifying your POM file as described in the documentation.

The Circle of Life(cycles): It Builds Us All


A very simple way to get started with Failsafe is simply to add the following to your POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.19.1</version>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
            </goals>
        </execution>
        <execution>
            <id>verify</id>
            <goals>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

This tells Maven to add Failsafe’s integration-test goal to the integration-test stage of your build when you run it, then the same for the verify goal.  This means that to run your functional tests, all you need to do is run Maven with a build goal of “integration-test” or later in the lifecycle, including the popular “mvn install”.  To skip your functional tests, simply pick a build goal prior to “integration-test”, such as “mvn package”.

Of course, this leaves you with the disadvantage that you won’t be able to deploy the application to any environments until you wait on all your tests to finish, and it probably won’t deploy the application until all your tests are passing.  If you want to use “mvn install” to deploy your application to your test environment without waiting on the functional tests to complete, consider using Maven profiles.

Separation Via Profiles


In Maven, you can construct different profiles in order to specify different ways you want a build to work, such as utilizing different plugins such as running Surefire (for unit tests) versus Failsafe (for functional tests).  Here is an example of what you would put in your POM to run Failsafe when the Maven profile with-functional-tests is specified:

<profiles>
    <profile>
        <id>with-functional-tests</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <version>2.19.1</version>
                    <executions>
                        <execution>
                            <id>integration-test</id>
                            <goals>
                                <goal>integration-test</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>verify</id>
                            <goals>
                                <goal>verify</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Notice that everything within <plugin></plugin> is exactly the same as in the first example; the only difference is the Failsafe plugin only runs when this profile is specified (on the command line with -Pwith-functional-tests).  It also gives you the benefit of limiting which environments the integration & regression tests actually run on, since developers won’t want to run every single functional test just to make sure the build succeeds before they can push changes to the code repository, and they won’t run unless they specify this profile explicitly (unless you put this under the “default” profile, and then they’ll just hate you :-P).

Annotation As a Solution


Yet another approach suggests creating an empty interface simply for marking purposes and then using that interface as a @Category to distinguish between your test types.

You might define a file such as IntegrationTest.java:

package com.test.annotation.type.IntegrationTest;
public interface IntegrationTest {}

And then use it in a real test as such:

import org.junit.experimental.categories.Category;
@Category(IntegrationTest.class)
public class RealTest {
    // etc...
}

You then need to set up the POM so that the Surefire plugin (for unit tests) explicitly ignores your IntegrationTest type:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.19.1</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.19.1</version>
    </dependency>
  </dependencies>
  <configuration>
    <includes>
      <include>**/*.class</include>
    </includes>
 <excludedGroups>com.test.annotation.type.IntegrationTest</excludedGroups>
 </configuration>
</plugin>

Also note the choice of surefire-junit47 for the artifact ID of Surefire, since this particular version correctly detects categories assigned with @Category.

Finally, you need to set up the POM so that the Failsafe plugin will actually run your IntegrationTest type (and only that type) during the integration-test build stage:

<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.19.1</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>2.19.1</version>
        </dependency>
    </dependencies>
    <configuration>
        <groups>com.test.annotation.type.IntegrationTest</groups>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>integration-test</goal>
            </goals>
            <configuration>
                <includes>
                    <include>**/*.class</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>

The downside to this approach is that you have to write that IntegrationTest.java interface file into each module you plan to use integration tests in.  If you have a multi-module Maven project, it violates the principle of DRY.  Plus, it seems to involve more XML (or at least more complex XML) than the previous methods, and dependencies on surefire-junit47 and org.junit.experimental.categories.Category that you wouldn’t otherwise need.

Article Sources:



Saturday, April 2, 2016

Arduino Day Special: Make an EPROM Tester with an Arduino Mega and Octal Latch

I could have just asked around to see if anyone had an EPROM validator, but why ask when you can spend several hours doing it yourself, and then several more hours writing in pedantic detail about it?  Of course, I must have the DIY bone...

Who still uses EPROMs, anyway?


While working on old solid-state pinball machines from the 1980s and late '70s, you might run into a situation where a dead ROM chip needs to be replaced.  Certain types of machines (I'm looking at you, all you Gottlieb System 80's) suffer a problem where coils can get locked on due to bad grounding design throughout the system, and then cause transistors and all sorts of other things on the driver board and even possibly the main board to fry themselves.  In other cases, battery corrosion might leech into the ROM chip and possibly compromise it.  No matter what the case is, you might find yourself in need of new ROMs at some point.

Now I could easily go and find new ROMs for my game, order them, and call it a day -- oh wait, I did mention System 80, didn't I?  Well it turns out Gottlieb (or the remnants thereof) is very picky about their licensing and who can sell related products, and the one legitimate source of the game ROM wants $50 for it.  I'm sorry, but I'm not paying that much.  I'll just get my own ROM chips and try to find a way to get the source code.

Now there are two things you need to do before plugging a new EPROM into a device:
  • Make sure it is erased
  • Program it with your new program
In both steps, you probably want to make sure the job was done correctly, no?  It would not be great to discover the program either didn't burn correctly, or couldn't burn correctly because there were already some zeros living on the EPROM that don't happen to line up with the zeros in your program.  Now again, I'll pose the question I asked to you at the top. ;)

Sanity Check


Before going down the rathole of doing this myself and having to do both the hardware setup and software programming (let's face it, wiring by itself takes enough time), I wanted to see if anyone had attacked this problem before.  I found, besides various forum posts that don't offer a complete solution, someone's GitHub code where they had utilized three different I/O registers on the chip to make this happen.  That's all fine and dandy, and was in fact the solution I was about to implement for myself... until I looked a little bit closer at the choice of I/O registers used and what the names of some of the pins were.

The ATmega2560 chip featured on the Arduino Mega happens to have outputs for /RD, /WR, and ALE.  I also noticed one register whose pins were labeled AD[7:0] and then another one whose pins were simply labeled A[15:8].  This evoked memories of my 8051 Microcontroller class in college (no, I swear I'm not that old yet!), and I realized this implies the chip can somehow multiplex its output of the first 8 address bits with the input (i.e. the data line) coming from the EPROM itself.  So, yes, it is in fact possible to use only two I/O registers on the Arduino Mega in order to read/write to an external chunk of memory. 

However, before you get started, note this approach requires access to a 74x373 or 74x573 8-bit octal latch chip whose timing specifications comply with the requirements mentioned on page 28 of the ATmega2560 datasheet.  The only difference between the 373 & 573 is the pinout, so use whichever you think will be more convenient for your end result (most people pick the 573 thusly).

Don't Forget To Register For This Service


I turned to the ATmega2560 datasheet and found the simple steps on how to do this.  In order to enable the chip to take total control of the PORTC (A[15:8]) and PORTA (AD[7:0]) registers plus the /RD, /WR, and /ALE signals so you don't have to worry about driving them yourself or changing input states, you need to be concerned with the two registers XMCRA and XMCRB.  These control the behavior of the XMEM (eXternal MEMory) functionality on various AVR chips including the ATmega2560.

Paraphrased from the ATmega2560 datasheet starting on page 36:

XMCRA has the following settings:
  • SRE (Bit 7): Set to enable the XMEM interface.  If you want to do anything described in this post at all, you must set this bit to 1.
  • SRL[2:0] (Bits 6:4): The Wait-State Sector Limit.  If you are worried about valid data not being ready from your EPROM quickly enough given the clock speed of your AVR, you can add wait states, and even specify to a degree which addresses get what particular wait states.  For my case, I dictated that all of the external memory shall be governed by one single wait state, so I set SRL[2:0] to 000b.
  • SRW11, SRW10 (Bits 3:2): Wait State Select 1.  Since I am paranoid, I set these bits to 11b so it would enforce the maximum wait.
  • SRW01, SRW00 (Bits 1:0): Wait State Select 0.  Since I selected to use only one wait state, the value of these bits don't matter.
XMCRB has the following settings:
  • XMBK (Bit 7): External Memory Bus-keeper Enable.  When this bit is set, the chip will retain the most recent value seen on the bus, even when another device would have set the lines to high Z.  This means the address hangs around on PORTA after ALE goes low (normally the address would be wiped out as the bus goes high Z for just a bit before the data is driven onto the port).  Also, the data from the EPROM hangs around on PORTA after /RD goes high (normally it would get wiped out as the bus goes to high Z before the AVR writes the next address).  Basically it acts like a smart latch that you don't have to toggle yourself, and in fact, you can activate this feature on PORTA without necessarily using the rest of the XMEM interface simply by setting this bit.
  • Reserved (Bits 6:3): Leave these alone.
  • XMM2, XMM1, XMM0: External Memory High Mask.  These bits determine how much of PORTC is given back to you for regular GPIO use.  If you have a device smaller than 64K words, then obviously you won't need (and it probably doesn't even have inputs for) all 16 address lines.  For example, my 2764 chip (8KB EPROM, 8K words * 8 bytes/word = 64Kbits = 8 KB) only uses 13 address lines, so I can set these XMM[2:0] bits to 011b so that I can regain the regular use of PORTC[7:5] if desired to do my usual reads from sensors, driving robot controllers or LEDs, or other general shenanigans.
You can see how I finally chose to set these registers in the code example down at the bottom.  Later on, I will also describe the instructions you have to send to the chip in order to get it to read memory, including exactly how to send a memory address to the EPROM through A[15:0].

Another important caveat mentioned in the datasheet discusses exactly how memory addressing works.  Since the ATmega's own memory is addressed from 0 to 0x21FF, you can use the principle of aliasing to access the beginning of your EPROM.  Without aliasing, these bytes would be masked by the ~8KB of internal SRAM plus other MMIO/PMIO on the AVR.  Thus, to read the first 8,404 bytes of your EPROM, you need to actually start by reading memory address 0x8000.  Also, if you have a ROM whose size is >32K words (e.g. the 64512 EPROM chip), there are other special considerations you need to make as well.  This is explained in more detail on pages 31 & 32 of the datasheet.

Making Connections


Next up is actually wiring up everything on the breadboard to the Arduino Mega.  (You do remember I'm still using an Arduino despite talking about all the mumbo-jumbo from the ATmega datasheet, yes?)  The wiring diagram to use is shown on that datasheet on page 28, Figure 9-2.  Also note that the 2764 datasheet (at least the one I was using) mentions that its /G line should be hooked up to the /RD line of the memory controller (thus saving me from trying it on something else and being disappointed).  Also, when the ATmega2560 datasheet mentions that the latch should be transparent to the EPROM and/or AVR when G is high, that means that ALE on the AVR should be hooked up to /G on the latch, not /E, since you don't want the latch to ever output high-Z; it should either be propagating D (the latch input) through Q (the latch output) when ALE is high (i.e. what they mean by "transparent") or propagating through Q what the state of D was just as ALE was set low throughout the whole time ALE remains low.

Besides Figure 9-2, which you can open up for yourself, here' s a table of the same information:

MCULatchEPROM
/RD/G
AD7:0D7:0
AD7:0D7:0
Q7:0A7:0
A12:8A12:8
ALEG

And here's a picture of my final setup:




Assembled In the USA


Yes, a mark of quality indeed... Anyway, if you've gone this far, why not write a little bit of assembly code just to put your effort over the edge into ridiculousness?  Because I am lazy and I use Windows mostly for AVR development, I still use the plain ol' Arduino IDE and blend assembly with C code (also I think it's fun to fly in the face of all the haters of basic Arduino stuff).

The macro for running assembly code inside C is called asm(), and each line of assembly can go into a double-quoted string that can be chained back-to-back without commas (but doing multi-line asm() calls is a bit outside the scope of this post).  When you add the keyword volatile to it, that lets the compiler know these values are subject to change at any time and the command needs to be rerun with any new values that might have been loaded into the variables representing the arguments.  Without using the volatile keyword, you might run a loop from 0 to 32767 with the intent to access the ith element of the EPROM, but only ever access the 0th element of the EPROM because the compiler "optimized" the assembly to assume the address argument doesn't change.  Whoops!

I started with the instruction lds (Load Direct from SRAM) to fetch external memory.  It takes two arguments: a register (any one register from r0 to r32 will do) and a constant.  This constant must be hard-coded into your assembly statement and cannot be provided by a variable.  Unfortunately, this doesn't really facilitate testing unless you want to write a really long unrolled loop!

Fortunately, there are instructions in assembly that allow you to store the memory address into a register, read the memory address indicated by the register, and then post-increment or pre-decrement that number for you so you don't even have to worry about updating the index.  Specifically, registers R26 through R31 handle this.  The odd-numbered registers store the high byte of the 16-bit memory address, and the even-numbered registers store the low byte.  For a diagram, check Figure 7.5.1 on page 14 of the ATmega datasheet.  These six registers represent three 16-bit special registers called X, Y, and Z.  In my code, I use Y (r28 & r29) because it worked most reliably out of the three.

At Last... The Code!


Note: Be Sure you have selected the "Arduino/Genuino Mega or Mega 2560" board as your choice in the Arduino IDE, or else it will not load the appropriate header files and will complain that XMCRA and friends are undefined.

/*   Note: If you want to test the boundary conditions, 
 *    the last address of internal SRAM is 0x21FF and the 
 *    first address of external SRAM is 0x2200, which also
 *    actually corresponds to address 0x2200 on the SRAM.
 *    To hit the very first address of the SRAM (0x0), 
 *    you must take advantage of aliasing by reading from
 *    0x8000 to 0xA1FF.
 *    
 *    The following code demonstrates writing to an
 *    internal register and will fail to write to the first 
 *    available address of an EPROM:

  asm volatile("ldi r16, 0xFF");
  asm volatile("sts 0x21FF, r16");
  asm volatile("sts 0x2200, r16");

 */
uint32_t i;
volatile unsigned int c, d;

void setup() {
  XMCRA = 0b10001100;
  XMCRB = 0b10000011;
  Serial.begin(115200);
}

void loop() {
  delay(1000);  // this helps avoid garbage at the beginning
  /*
  // This part proves the auto-increment feature is working
  // and that the first 10 bytes are indeed being read correctly
  asm volatile("ldi r28, 0x00");  // YL
  asm volatile("ldi r29, 0x80");  // YH

  for (i = 0x8000; i < 0x800A; i++) {
    asm volatile("sts (d), r28");
    asm volatile("sts (d + 1), r29");
    Serial.print("Contents of address ");
    Serial.print(d);

    asm volatile("ld r0, Y+");
    asm volatile("sts (c), r0");
    Serial.print(": ");
    Serial.println(c, HEX);
  }
  */

  asm volatile("ldi r28, 0x00");  // YL
  asm volatile("ldi r29, 0x80");  // YH

  for (i = 0x8000; i < 0xA000; i++) {  // for an 8KB EPROM
    asm volatile("ld r0, Y+");
    asm volatile("sts (c), r0");
    // The following prints out hex in the format
    // FF FF FF FF  FF FF FF FF  FF FF FF FF  FF FF FF FF
    if (c < 16)
      Serial.print(0);
    Serial.print(c, HEX);
    Serial.print(" ");
    if (i % 16 == 3 || i % 16 == 7 || i % 16 == 11)
      Serial.print(" ");
    if (i % 16 == 15)
      Serial.println();
  }

  while (true) {
    // spin lock
  }
}


Reference Materials


This article would not be possible without the help of the following:

ATmega2560 Datasheet
AVR Instruction Set Manual
Introduction to AVR assembler programming for beginners
GCC inline assembler cookbook