Skip to content
Will Paul edited this page Dec 14, 2015 · 20 revisions

Storm Server Wiki

Structure

Data flow through the application

Application data flow

Technology stack

Technology stack

Directory setup

/api
   /constants   # Constants shared across server & client
   /controllers # HTTP RESTful controllers, 1-1 file-controller relationship
   /handlers    # WebSocket equivalent to controllers, 1-1 file-handler relationship
   /models      # Mongoose models
   /services    # Shared logic
  app.js           # Initial Express setup 
  routes.js        # Express-Enrouten hooks up controllers to routes
  dispatcher.js    # Routing equivalent for Socket.io
  event-stream.js  # Extension of EventEmitter to decouple the realtime handlers from the Socket.io implementation
/test
   /unit
index.js        # Starts runtime Babel and app.js

API

Operations that take place in a room are handled over Socket.io in realtime, everything else happens over a RESTful HTTP API. To decouple the business logic from the transport layer we made our socket handlers be triggered by an internal event emitter stream. To ensure the correct listeners get invoked all event identifier strings are stored in a constants object that is passed to the client for use.

Routing is being handled by Express-Enrouten from Paypal allowing our controllers to be thinner and better organized. Commonly used actions are stored in services and shared across handlers and controllers.

To prevent the services from getting bloated and to reduce coupling, model focused actions have been moved into the models themselves as static methods.

Database

The Mongoose library was utilized to interface with our Mongo database. The Board model contains any information directly related to a Board including Users, Admins, and name. However, Ideas and Idea Collections are stored separate from the Board to reduce nested associations which can complicate and add bulk to queries later on. A generated unique string serves as a board id used to link Ideas and Idea Collections to their respective boards. Due to lacking an explicit association cleanup of Ideas from a deleted Board is left to us and is handled in a post remove hook. Pre and post hooks on saves helps to ensure more complex validation in Mongo.

Build process

We use Babel to get the latest ES6 features, as well as ESLint with a slightly modified version of Airbnb’s configurations. Mocha is being used as the testing framework, with Chai for assertions, and Sinon for stubs and spys.

Post-mortem

Technical Choices

By far our biggest mistake was starting out with an end-to-end framework (Sails). By putting all our eggs in that basket it made it very hard to deviate from the path that the people at Balderdash had laid out. While painful, the conversion from Sails to Express, Sails.io to Socket.io, and Waterline to Mongoose, gave us a substantial performance boost and a lot of freedom.

Things that went well

  • The code base is actually in pretty good shape. The combo of enforcing a very strict linter and our code review process kept the spaghetti code away.

Things that didn't went well

  • Our unit testing coverage, while improving, is a lot lower than we would like.
  • We didn't get our code done in time for client to integrate, and we didn't do a good enough job communicating to them so they could be prepared (stubbing out our calls).

The future

  • Mongo is fast, but we query it a lot of times per second, especially as it scales to more users. How we planned to handle this was by caching all the queries in Redis (probably with cachebox).
  • We didn't get to private rooms and dashboards.

Details

Repo Links