Suggestion for auto-login with deployments

[tyler-dunkel]

Hi, love the package btw! I am using it to offload some cpu heavy tasks to a node server and was wondering what would be your suggestion as far deployment goes. I currently deploy 2 meteor app servers using mup. However, now i plan on using jenkins or another tool to deploy the node server that handles these jobs at the same time. The only thing i was wondering is how to automate the login portion so it doesnt prompt in the terminal when the server is started. once again thanks!

[vsivsi]

Hi, anytime you configure a "cluster" of machines working together (I use that term in the loosest sense) you will face issues of how to manage and coordinate them, and one of those issues is authentication. Meteor's authentication packages are of course designed primarily with user authentication in mind, and so this package tries to fill some of the gaps to allow more autonomous agents to also become authenticated.

Given that, I'm not sure there is any one right answer for how to distribute such credentials securely. One approach is to somehow securely bootstrap each worker on creation with a valid Meteor authentication token which is then stashed in the worker's process environment. Authentication tokens don't typically expire on server restarts, so as long as workers usually don't live longer than authentication tokens, you are golden. If workers do live longer than tokens, then you either need to change the token expiration policy, or build a secure mechanism that allows authenticated workers to periodically receive a new token before the old one expires.

ddp-login allows authentication of DDP connections using just a token, both on the command line (via an environment variable e.g. METEOR_TOKEN) and using the API 'token' method or equivalently by calling login.loginWithToken(...).

Of course all of this is necessary because it is inadvisable to hard code user/pass information in a worker...

Hope that helps.

[tyler-dunkel]

hmm i see there is a login with email function as well. could i pass the email and pass as node env variables and then login that way? Would that work as long as the env vars were set correctly every time?

[vsivsi]

It would, but tokens are probably a better idea simply because they expire and are easily revoked in case of compromise.

[vsivsi]

Clarification: This package will not access email/pass from the environment for you (as it does with tokens), if you stash such credentials in the environment, then you will need to retrieve them yourself and pass them through the API in the normal way.

[tyler-dunkel]

oh yea i understood that part. something like DDPlogin.loginWithEmail(ddp, process.env.authEmail, process.env.authPass, function(err, usr) {}); but I will probably end up going the token route. Thanks! just one more question if you don't mind, I plan on starting up a cluster of these workers to get full usage of my server. Is there anything as far as meteor-job's processJob-style workers that would make this more complex than simply starting up a load-balanced cluster with something like pm2? Once again thanks for you help!

[vsivsi]

Hi, I'm only somewhat familiar with pm2, but my understanding of such systems is that they load balance incoming requests (such as HTTP connections) among some collection of application instances.

In contrast, job-collection workers pull work from the server when they are ready (or via polling or observe based notifications when idle). So if there are more workers than jobs, the work will be distributed semi-randomly among the workers. If there are more jobs than workers then near 100% utilization of workers should be possible. Scaling under heavy workloads is as simple as provisioning more worker processes when more jobs are becoming ready to run in the job-collection than are being completed (and vice versa).

[vsivsi]

BTW, if you'd like to see a simple worker script that does everything including token based authentication and graceful handling of shutdown signals, check out the playground worker:
https://github.com/vsivsi/meteor-job-collection-playground-worker

[tyler-dunkel]

yea thanks! I checked out the playground worker. Could you explain why you use processJobs as well as observing the changes on the collection? are both necessary to pull work in continually? as far as pm2, yea the server won't have any incoming requests so that part of their load balancing would do nothing. But it starts 4 (default) instances of the app so i was wondering if those 4 instances would all be separate workers pulling work or if there would be some issue with that but from what I can understand from your answer I don't think there will be!

[vsivsi]

Observing changes is not necessary, but if you use it you can avoid polling. It decreases latency when the workers aren't being 100% utilized.