Cleanup withdrawer code

[awilliams1-cb]

As noted in #17 there are some spots where the code can be cleaned up / better optimized. This work should be done as a fast-follow.