A New Start

traveler

After about three and a half years at Deutsche Bank, this week I start a new job at Interactive Intelligence, working in its cloud platform and family of products.  Its great to be back at a tech company.   Especially one that’s doing really cool stuff with web technology, and that has a really innovative culture.  The bank was great to me while I was there.  I worked with some great people and we overcame some difficult challenges.  I got to a point where my passions in software development diverged from the work I was doing in my day job at the bank, and I knew it was time to take the next step.

The first thing I’m working on is an EmberJS application so I’m excited about diving in, not only to the ember ecosystem, but other concepts such as ES6 new CSS concepts like flexbox as well.  There are some really cool things about the culture of the team, such as participation in meetups, hackathons, and open source, all things I look forward to partaking in.

My side projects and goals are still on, though they’ve slowed considerably in the job transition of the last few weeks.  I’m looking at my habit forming reactJS app still, and plan to post here as it is made.

Right now I have a bit of a fire hose of info to learn and become proficient in, but its fun and challenging, just how I like it B-)

A Couple of Essentials for Your AngularJS Unit Testing Toolkit

Toolbox_icon_transparent_backgroundHere’s a couple of AngularJS features to help with unit testing controllers that I was able to take advantage of this past weekend.

Loading json test data from a file:

You can use the jasmine-jquery library for a simple api for loading json from a file (among other useful features.  For a controller, you can use as follows…
//you have to use "base" to address the root of your project
jasmine.getJSONFixtures().fixturesPath= 'base/data';
peopleData = getJSONFixture('people.json');

Simulating http response data with $httpBackend:

Once you have json data loaded, you will want to supply it to the $http process so that you can control the data that comes back.  You can do this through through the $httpBackend object
$httpBackend = $injector.get('$httpBackend'); 
$httpBackend.whenGET('/api/people').respond(200, peopleData);
or to simulate an error:
$httpBackend.whenGET('/api/people').respond(500, '');
and since the $http calls are asynchronous, you can simulate the server responding by calling:
$httpBackend.flush();
You can use these json file load and $httpBackend together to create a nice, easy way to stub json response data.

Further info for your reading enjoyment…

Anatomy of my MEAN App

anatomyOver the last couple of months, I’ve been working on an app using the MEAN stack to track strength over time. Ironically, I’ve been spending my after-hours time tinkering in JavaScript technology instead of exercising, so I’ve gotten a bit out of shape :-P

Before focusing on my next off-hour project, I’d like to give a rundown of some of the more interesting parts and lessons. I’ve posted some snippets, but you can see it all together on strength tracker’s github page.

First of all, here’s what it looks like:
StrengthTrackerFinal

Starting Quickly with Angular Seed

I used the github project angular-seed to get me going. It sets up the basic components you need to develop an angular app, such as jasmine and protractor for tests, and puts out some very basic components such as a directive (for showing version), and a couple of basic controllers that just show text that you can navigate between. Just clone the project, install the required dependencies with npm, and your are good to go.

For adding the rest of the stack (Mongo, Node, Express), I just used the express generator, which laid out the express directory structure and files to get a basic server running. I then copied my angular source tree into the “public” directory.

I played with mean.js, and used its yoeman generator to create a full stack skeleton, but I found that it gave me much of what I wanted to learn how to do myself, such as authentication with passport and connecting to mongo. Therefore I scrapped it for this project, though I think its a good option for getting started on a MEAN app quickly.

Authentication with PassportJS

I wanted to add some really base user registration and authentication, so I used PassportJS. Passport provides a plugin authentication architecture with some existing implementations such as basic userid/password, Twitter, and Google. While passport provides value by giving you multiple options for authentication, I’d pass on it in the future if I’m just doing local basic userid and password authentication. Most of the things I did in this stack ended up being fairly easy, but the couple of times I got hung up for an hour or 2 was with passport. I found 2 areas of difficulty.

First is that passport requires a specific order of loading its various parts interwoven within the larger node startup. If anything is out of order, then your authentication or server startup fails with unclear errors.

app.use order:

  • Express basics(express, cookie parser, bodyParsers)
  • session(with the right sauce)
  • passport(init and session)
  • authentication route(with passport strategy setup)
  • other routes

Express Setup with PassportJS – Order matters!

app.use(logger('dev'));
app.use(express.static(path.join(__dirname, 'public')));
app.use(cookieParser());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: false }));
app.use(session({
  secret:"keyboard cat",
  resave:false,
  saveUninitialized:false,
  cookie: {
    maxAge: 60000
  }
}));

app.use(passport.initialize());
app.use(passport.session());
//the auth route includes passport strategy setup, which as to happen before
//other routes, and after passport initialization
app.use('/auth', auth(passport));
app.use('/exercises', exercises(passport));
app.use('/users',users(passport));

Secondly, I had some initial trouble interweaving the functions needed for session persistence of logins. To login using session persistence, you need to add deserialize and serialize methods. These will interplay with calls you make to the server and with your login.

  1. route handles login post, calls passport.authenticate
  2. authenticate calls the login strategy function you provide to determine if the user exists and can login given the submitted credentials
  3. callback given with the authenticate call is invoked, sending the appropriate response, 401 vs successful response and payload.
  4. user id is serialized into the session so future calls can be authenticated (until the user logs out or the session expires.
  5. the login function that passport attaches to the request gets invoked, user serialization call is invoked
  6. finally the function that you provide to the rest login gets invoked. If you see that the login is successful (assuming you’ve called the “done” function with the right stuff), then you can process that result and send the appropriate data / response code.

Here’s the code. Could be refactored to make thing’s clearer, but this gives you an idea of the complexity

router.post('/login', function(req, res, next) {
      passport.authenticate('local', function(err, user, info) {
        if (user) {
          req.logIn(user, function(err) {
            if (err) {
              throw err;
            }
            return res.json(user);
          });

        } else {
          res.sendStatus(401);
        }

      })(req, res, next);
  });

  router.post('/logout', function(req, res, next) {
    req.logout();
    return res.sendStatus(204);
  });

  passport.use(new LocalStrategy(
    function(userid, password, done) {
      db.collection('users').findOne({userid:userid, password:password}, function(err, result) {
        if(err) {
          throw err;
        }
        if (result) {
          done(null, result);
        } else {
          done(null, false);
        }
      });
    }
  ));

  passport.serializeUser(function(loggedInUser, done) {
    done(null, loggedInUser.userid);
  });

  passport.deserializeUser(function(id, done) {
    db.collection('users').findOne({userid:id}, function(err, result) {
      done(null, result);
    });
  });

End to End Angular Tests with Protractor

Protractor is an end to end testing framework that combines Jasmine and WebDriver. It operates similarly to cucumber, plus it has some AngularJS specific APIs. I was able to quickly get login functionality and some navigation and DOM verification working. I also created some data setup/teardown javascript objects so that users and workouts exist for the test and are cleaned up after.

The setup/teardown code was fun (in a geeky way). I created state objects for my two main data structures in the app: users and exercises. The state object simply takes a json object of whatever data I want to set up and implements database setup and teardown for that object:

var userState = function UserState(user, dbClient) {
  return {
    setup:function() {
      dbClient.insert('users', user, function(result) {
        console.log('inserted test user');
      });
    },
    teardown:function() {
      dbClient.remove('exercises', {userid:user.userid}, function(result) {
              dbClient.remove('users', {userid:user.userid}, function(result) {
                console.log('removed user %s, and all associated exercises', user.username);
              });
      });
    },
  };
};

Then at runtime, I call setup and teardown before and after the test/tests. In this case, I set up and tear down my user before and after all tests are run:

  beforeAll(function() {
    userState.setup();
    exerciseState.setup();
    browser.get('index.html');
  });
  afterAll(function() {
    userState.teardown();
  });

You can see it all together in my end to end tests directory.

Tight Feedback with Gulp and NodeMon

One thing I really enjoyed about getting back into the javascript ecosystem is how quick I got feedback on my changes. To avoid having to restart node when I made changes, I installed nodemon, a node server that will detect changes and restart itself. Additionally, I started using gulp to lint and execute unit tests as I make changes. It was extremely easy to set up a watch to look for code changes and run unit tests and lint immediately.

Here’s all the gulp code you need to get continuous linting with jshint and testing with karma:


gulp.task('lint', function() {
  return gulp.src(files).
    pipe(jshint()).
    pipe(jshint.reporter('jshint-stylish'), {verbose:true});
});

gulp.task('test', function(done) {
  karma.start({
    configFile: __dirname + '/karma.conf.js',
    singleRun:true
  }, done);
});

gulp.task('watch', function() {
  gulp.watch(files, ['lint', 'test']);
});

Easy Charts with Chartist

Chartist is a library build on top of D3. I saw a conference talk on it and it looked interesting, so I used it for my charting component. It already has an angular directive, so getting the charts to show up was just a matter of setting some simple json to the scope. It has databinding so the chart updates magically as you update values in the UI.

Styled Web Widgets with Angular Bootstrap

I wanted to use some bootstrap components, such as its alert box, dropdown and dialog box. I was able to leverage existing bootstrap angular directives.

Alerts

Alert

Modals

Modal

Dropdowns

Dropdown

Modular JUnit Behavior Tests

Over the years I’ve developed a way of doing unit tests in java that’s a bit different from folks I work with.  I’ve pieced it together from various books I’ve read on unit testing, and from from experience of writing a bunch of these.

I’ll walk through an simple example with FizzBuzz to show what I’m talking about, because explaining won’t do it justice.  Note:  This is necessarily not the optimal way of TDD for FizzBuzz.  I’ve used it just as a simple example for showing the pattern.

What I do is split the tests into the classic 3 step pattern of Arrange, Act, Assert, doing this with a given/when/then language.

I Arrange with a factory method

public FizzBuzzRunner givenAFizzBuzzCalculator() {
    return new FizzBuzzRunner(new FizzBuzzCalculator());
}

With more complicated setups, I’ll augment the factory method by having it take a builder pattern.  Then you can say things like

givenAFizzBuzzCalculator(withX(X).withY(Y).withZ(Z));

I Act with a runner class

private class FizzBuzzRunner {
    private final FuzzBuzzCalculator fizzBuzz;
    public FizzBuzzRunner(FizzBuzzCalculator fizzBuzz) {
             this.fizzBuzz = fizzBuzz;
    }

    public FizzBuzzVerifier whenFizzBuzzRunsFromTo(int from, int to)     {
         return new FizzBuzzVerifier(from, to);
    }
}

I Verify the result with a verifier class

private class FizzBuzzVerifier {
    private final List<String> fizzBuzzResult;
    public FizzBuzzVerifier(List<String> result) {
          this.fizzBuzzResult = result;
    }

    public void thenVerifyExpectedTextAtIndex(String expectedText, List<Integer> expectedIndices) {
          for(int i=0;i<expectedIndices.size();i++) {
               assertThat(fizzBuzzResult.get(i), is(equalTo(expectedText));
          }
    }

     public void thenVerifyIndexNumberTextAtIndex(List<Integer> expectedIndices) {
          for(int i=0;i<expectedIndices.size();i++) {
               assertThat(fizzBuzzResult.get(i), is(equalTo(String.valueOf(i));
          }
    }

}

I Daisy Chain the testcases

@Test public void
shouldReturnFizzOnMultiplesOfThree() {
    givenAFizzBuzzCalculator().
          whenFizzBuzzRunsFromTo(0, 30).
          thenVerifyExpectedTextAtIndex("Fizz", Arrays.asList(3, 6, 9, 12, 18, 21, 24, 27));
}

@Test public void
shouldReturnBuzzOnMultiplesOfFive() {
    givenAFizzBuzzCalculator().
          whenFizzBuzzRunsFromTo(0, 30).
          thenVerifyExpectedTextAtIndex("Buzz", Arrays.asList(5, 10, 20, 25));
}

@Test public void
shouldReturnFizzBuzzOnMultiplesOfFifteen() {
    givenAFizzBuzzCalculator().
          whenFizzBuzzRunsFromTo(0, 30).
          thenVerifyExpectedTextAtIndex("FizzBuzz", Arrays.asList(15, 30);
}

@Test public void
shouldReturnTheNumberOnNonMultiplesOfThreeOrFive() {
    givenAFizzBuzzCalculator().
          whenFizzBuzzRunsFromTo(0, 30).
          thenVerifyIndexNumberAtIndex(
             Arrays.asList(0,1,2,4,6,7,8,11,13,14,16,17,19,22,23,26,28,29));
}

I really like the approach because it has a few benefits…
  • Readability – The testcases themselves work as a DSL.  You can understand what the test is doing very easily.  It also breaks the code that runs the test into several objects, with like methods grouped together.
  • Reuse – Since my test logic is built into objects, I can reuse these for multiple tests.  I also try to write the execution methods in a way where I can pass in the variations in data.
  • Speed – Because the test lend themselves to more “DRY” programming, especially in more complicated scenarios, I’m writing less code.  Also, the objects follow a specific order, and each method returns the next object in the list.  This is awesome for command completion.  Imagine after typing your given method, you get a command complete menu of your “when” methods available.

Wrapping Up Strength Tracker

giftWrappedOver the past couple of months on my spare time, I’ve been creating a MEAN stack app to track my strength training called strength-tracker.  I’m at a point where I’ve got the functionality working, as well as some super basic login and registration capability.  Before I move on the to next web framework, I’d like to add some finer points to the app to get some exposure to how to do these things work in the JavaScript and Node ecosystem, as well as to give myself a little better feeling about what I’ve done.
I’m looking to plow through these things during the evenings this week, and then post something on lessons learned about MEAN stack programming from the project.  Here are my wrap-up tasks:

Improve the architecture

Because I’m new the MEAN stack.  Much of the app was written in a mode of fiddling with the code and frameworks to get it going.  Now that I got things functional I want to do some refactoring modularize things and separate concerns.  A few of things I have in mind:
  • Use node’s require mechanism to split out some concerns on the node-side, such as passportjs code.
  • Use AngularJS’s “controller as” construct, get away from the $scope

Add More Tests

Of course to avoid making my refactoring a painful exercise I want to add tests around my code, I’ve been doing this on the AngularJS end of things, but I also want to get some automation going on the node/express end.  Plus this will give me a good intro to in testing node.

Add Error Messages

Since I’ve been pushing to get this project working, I’ve been intentionally putting off error display.  Now’s the time to circle back and errors are properly communicated to the user when appropriate.  I’ll likely implement an animated interstitial for it, and I’ll probably find something I can use from the angular open source community.

Use Grunt or Gulp to set up a nicer build solution

I hear Gulp is the newer hotness but it seems like grunt has a more goodies as far as plugins  out there. I’d like to get an automated build going based on changes as I go, and include things like jshint/linting, tests, coverage, complexity, etc.

Use a require mechanism in angular

Right now the code base is loading all its script dependencies in the main index.html as script tags.  I’d like to try out one of the javascript libraries for including file dependencies, like requireJS or browserify

Taming Software Code Complexity and Technical Debt

bearAndHandlerThe project I’m currently on has a good deal of complexity.  Its built on a 10+ year old codebase that has passed through multiple development groups with varying levels of skill and pressures.  It can cause some problems around the team’s ability to deliver new functionality with speed and quality.
I’m currently in the midst of working with the team to get our team on the path of making the project less complex and more maintainable over time.  Here’s our approach:

Schedule Regular “Debt Payments”

If all new development goes into business driven features, and the complexity and cost to adding those features continues to rise, your project grinds to halt, hurting everyone.  Therefore its a good practice to bake in “debt payments” into your release planning, mean a certain percentage is reserved for improving your codebase.  This time is dedicated to things like driving down static analysis issues, refactoring brittle parts, building in automation, etc.

Keep a Debt Payment Backlog

Just like the business keeps and prioritizes a feature backlog, the development should keep and prioritize its debt backlog.  Just like team meets regularly to “groom” the business story backlog, the dev team should meet regularly and refine its technical backlog, so that it makes the most of its debt payments.  Care should be taken with this backlog and changes to the codebase should be made responsibly and socialized with business stakeholders.  They like to hear things like what is the risk involved with a change, and what tangible benefit will it give the product.

Socialize Architectural Direction and Patterns

Part of what got us into a bit of trouble in the codebase was that we inherited a some less than well-architected codebase.  It had a set of patterns that were less than ideal, and when developers went in to add new functionalities, they copied how things were done for the sake of consistency and ease, propagating the patterns.  Additionally different developers might have different ideas of how the same codebase should be developed.  This phenomenon of propagating anti-patterns and disjointed coding will continue unless the team gets on the same page.  What has helped our team is coming together regularly and talking about the various inherited parts of the system and coming up with a shared understanding of where we want that architecture to go.

Work Effectively with Legacy Code

The Boy Scout rule and Broken Windows theory are important concepts to build into your team culture.  If we fix a bug or add a new piece of functionality to an existing piece of code or develop something new, we want make sure that the code has no static analysis issues, has low complexity, and is covered by tests.  Michael Feather’s book on the subject is a great read for coming up with ways to effectively change existing code.

Understand and Manage Pressures

What I mean by this is that the development team needs to be in charge of what they deliver.  I’ve been on too many projects were quality was sacrificed for initial delivery speed, and this is what I have come to:  If we as developers are going to sign our name to something (which we do when we commit it to source control), it needs to be done with quality and care.  Its a professionalism issue and needs to be baked into a team’s culture.  This requires an ability to be honest with yourself and with your stakeholders, to be able to negotiate, and to be able to say no when you need to.

Getting on a path of creating clean and maintainable code can bring new purpose and life to a development team.  Its great to be on a team that is proud and in control of what they do.

A Simplified View of MV* Web Pattern Differences

I was asked the other day about the difference between common web architecture patterns and I stumbled around a bit explaining.  After thinking about it some I came to the conclusion that I’m much better sketching these types of things out, so I figured I capture this in a post.

Much of my understanding of the difference between web patterns comes from Martin Fowler’s post on the subject.  While its a fascinating read, it does take a while to read and digest.  Hopefully my rudimentary boxes and arrows can give a fairly straightforward depiction of the differences.  Solid arrows indicate dependency direction, and dashed arrows signify data binding.

MVC (Model View Controller)

Example Framework: Backbone/Marionette
Highlight: Databinding between view and model
mvc-boxes

MVP (Model View Presenter)

Example Framework: GWTP
Highlight: Presenter handles all presentation logic

mvp-boxes

MVVM (Model View ViewModel)

Example Framework: KnockoutJS/Durandal
Highlight: Databinding between View and ViewModel

mvvm-boxes

So what’s Angular?  Angular describes itself as an MVW framework (Model View Whatever), as do others.  You can set it up to do any of the 3, but if you use its advertised features, especially the databinding between html and the scope object would suggest its more aligned with the MVVM school of thought.