Add support for RUN layer?

[weichunstevelee]

Looking at the current set of tools in containre_config, there isn't a way to add RUN layer. I skimmed through the code and figured it won't be too difficult to add one (?) but wonder what's your thought on this? Maybe you have considered in the past?

cc: @ianoc

[weichunstevelee]

Reading some previous notes i seem to understand why we couldn't have RUN, probably due to potential side effects? still cruious about your thoughts

[ianoc]

Sorry for the length of this, its a good question and maybe should just be a FAQ in the readme. But there's a few parts to this:

This stuff really produces OCI Layers, not something 'docker'. RUN is actually that i'm aware only really a pure docker concept. OCI layers are really just tar files + metadata.

Doing it? Probably not, which is a consequence of the above:

1. To RUN something, you need to run docker. Which is a root level daemon. So its not hermetic, and not something you can really know is around/in the environment. So its pretty much a giant pain to say run anywhere that might already be in a container (most cloud env's), CI sometimes even since nesting containers can be a thing, and on mac laptops since you need docker desktop.
2. In order to RUN something the general contract that i'm aware of is that it should be able to see and interact with the file system. But this code and its parent of rules_minidock explicitly avoids ever needing to gather and materialize all of the layers. Since the base layers can be both large and change relatively frequently. It can mean a lot of large downloads. -- RUN is afaik effectively "materialize the file system. Run the command. diff what the filesystem looks like"
3. (1) + (2) mean that you would need to do the RUN in a repository rule (or work around/ignore sandboxing and then sort of be just running stuff against the docker daemon -- i think the rules_docker ones do something like this). Which then means caching isn't nearly as robust (not remote, not really shared). And you can't use tools you build inside the repo in the RUN statement. Which sort of i guess defeats the point maybe?

Even if you break the sandboxing to run it, the tooling not being hermetic, and not installed/available everywhere means most folks then have to aggressively control how many docker images are used (they are pretty darn slow, and don't use bazels remote cache), and build/do them only in certain deploy environments where they can access that docker socket.

I'm not sure what you want to do, but there are maybe 2 alternatives worth looking at:

1. We mostly just baked anything that needed to be run into the BASE image
2. If you need to run some of your code and unpack it, building a rule/setup to run some code to output into a directory and then gather those results into a tar might be good enough? (There are probably some extensions here about trying to gather some particular input tars to give an initial base directory structure to mutate)

[ianoc]

If you can come up with a some pattern that would work for you and not be trying to drag docker in then i'm down to look at/help you explore it.

Otherwise, i think for the reasons above its not a good idea to do it per say. But if your team/folks are happy with the limitations/impact above you could make a new small rule set hooked into rules_minidock itself too that does this sort of operation i think. (Mostly in the rules since we try avoid it you would need to wire some of the publishing logic into another rule/step command of yours such that you can expose the layers built in the repo or externally to the local docker daemon)