init system for startup and interactive?

[yajo]

Is your feature request related to a problem? Please describe.

Exiting nix develop shell leaves running processes.

Describe the solution you'd like

For interactive and startup sections, it would be great to be able to leverage some small init system that automatically reaps child processes.

Describe alternatives you've considered

Adding traps like here, but still not easy to get going right always, and some programs don't support daemon mode out of the box:

devshell/tests/extra/services.postgres.nix

Line 20 in 7033f64

trap "pg_ctl stop" EXIT

Additional context

[zimbatm]

What method do you use to start postgres?

Usually, I have a Procfile that starts postgres, the backend and the frontend, and executed by foreman or hivemind. That lives in a shell of its own.

[yajo]

The plugin.

[hugosenari]

For files I created a files.services option using s6.

If there is any interest, I can port it to a stand-alone version and open a PR.

Basically,

• it adds s6 to devshell packages and startup
• for every files.services.{name} = true
  • create a $PRJ_DATA_DIR/services/{name}/run # start script
  • create a $PRJ_DATA_DIR/log/{name} # logs dir

Is s6, the best tool for the job? 🤷
I would be happy to hear about other options, comparisons, etc.

When we should stop?
Since devshell makes my env seamless, ⁣:love_letter: thank you all :love_letter:, I always have a shell open, but there is no 'main' shell terminal (tab, pane, window), I just close them and open another.

Know bug:
Autostart isn't default, because auto start s6 with direnv blocks first shell, but doesn't happen after shell has been initialized

[yajo]

After thinking more about this, maybe the fact of opening services just by entering the shell is a wrong concept. Yes, there can be multiple shells. Also it makes it difficult to to think about direnv integration and other shells.

I mean:

• If I use direnv, I can use Fish and get all context, but no services.
• If I use nix-shell, I get services but no Fish.
• If I use multiple nix-shell, PID get messed up.

So maybe all we really need is to define services and provide a service manager that you execute manually. Let's say it's called devshell-services.

So, you enter the folder, direnv does its work, you keep using Fish (or your favorite shell), and you just run devshell-services and have it going. You can add a task to your IDE that runs nix-shell --run devshell-services to start background services automatically. Etc.

I haven't tried yet with devshell-files. Does it allow this workflow? Does it integrate properly with the postgres service defined in this project?

[hugosenari]

Does it allow this workflow?

Is the standard workflow, but devshell-services is called initSvcs, there is an option to enable auto start but it freezes your first shell... The major bug I've found isn't start services but stop them with stopSvcs.

Does it integrate properly with the postgres

I never tried, I'm running Postgres as docker service, there is no such thing as integration, you just need to specify a service like. files.services.postgres = true; and initSvcs will run 'postgres' as start command and 'postgres-finish' as stopping command.

In most of cases we need more arguments for init, usually, I do something like this (one of actual configurations):

{
  # files.alias is a helper to devshell.commands = [ { name = "apiSVC"; command = "cd ..."; } ];
  # API
  files.service.apiSVC      = true;  # it configures s6-suverpisor files for apiSVC
  files.alias.apiSVC        = ''cd api; exec docker compose up'';  # stdout goes to .data/log/{name}/current
  files.alias.apiSVC-finish = ''cd api; exec docker compose stop'';
  # frontend
  files.alias.webSVC    = "cd webapp; exec npm run serve";
  files.services.webSVC = true;  # it configures s6-suverpisor files for webSVC
  # the actual config has also ngrok (to espose 8080 to the world), 
  # a docker api logger, because stdout of docker compose up have log for both api and postgres
  # my workflow involves a cloud workers, I created a service notify status with 'notify-desktop'
  # I could create a 'pomodoro' service to notify start and stops
}

I'm not sure how does it translate to TOML but should work since no function were called ;-)

And again, I'm using s6 and not sure if it is the best tool for the job, recommendation are welcome
Is far from perfect (like stop bugs) but works

edit: requires exec

[yajo]

I've experimented with it and it looks good, but s6 is indeed giving some headaches. cruel-intentions/devshell-files#5 fixes some of them, but still not perfect. For example, if running a postgres inside my IDE, exiting the IDE will remap s6 to be direct child of PID 1.

I'm attempting with tini, things seem to be more smooth.

[hugosenari]

if running a postgres inside my IDE, exiting the IDE will remap s6 to be direct child of PID 1

As I can see our cases are different,

• You start it from IDE expecting it to die with IDE.
• I start it from shell hoping that it would not die even if I close that terminal.

Anyway there are some bugs, running stopSvcs doesn't always stop all process.

If it worked, and you experienced some stop issues, we are on same page :)

There are two possibilities:

1. I'm not using it correctly
2. S6 isn't designed for our use cases
3. Off by one errors.

I'm attempting with tini, things seem to be more smooth.

Thank you, I will look at it.
Today, when thinking about how to make it autostart without block shell, I considered systemd-run --user initSvcs, to make systemd start s6, but if I will use systemd why not just use it systemd for anythiing else 🤔

Curiously, I'm trying to an option to intentionally make it child of pid 1 and don't block direnv, while you're saying that unintentionally it is child of pid 1.

[yajo]

As I can see our cases are different,

Yes, so it seems. I borrowed some ideas from here and there but I'm now using just devshell + a couple of customizations based on tini. It's making me very happy now, and I think the main specific point of enhancement in this issue is what I already explained in #170 (comment).

Now that I managed to not start services when doing nix-shell, I get proper direnv integration and services autostart when the IDE is asked to do so, no matter if 0, 1 or 2 shells are open. 😊

I opened #194 to explain how I managed to do it, to avoid going more off-topic here.

[oneingan]

I'm happy you choose s6 because is feature complete and wide compatible. But I will more happy if somebody gets to integrate https://github.com/svanderburg/nix-processmgmt into devshell.

[hugosenari]

Summary:
mlvzk
#35 PR using tmux
blaggacao
#47 suggest glorious
#72 similar issue
Suggest Nix RFC 0078
zimbatm
Is using foreman or hivemind
hugosenari
Is using S6 or S6-RC
yajo
Is using tini
uningan
Suggest nix-processmgmt

[hugosenari]

chvp
#253 PR using Honcho

[hugosenari]

When we should stop?
Since devshell makes my env seamless, 💌thank you all 💌, I always have a shell open, but there is no 'main' shell terminal (tab, pane, window), I just close them and open another.

Yesterday I was playing with this like:

• Every time we run activation script (config.devshell.startup) I link /proc/$PID/comm to $PRJ_DATA_DIR/procs/$PID

SESSION_PID=$$
PARENT_PID=$PPID
while grep -q direnv /proc/$PARENT_PID/comm
do
  SESSION_PID=$(ps -o ppid= $PARENT_PID|tr -d \[:space:\])
  PARENT_PID=$SESSION_PID
done

ln -s /proc/$SESSION_PID/comm $PRJ_DATA_DIR/procs/$SESSION_PID &>/dev/null

• There is a service that remove broken links and stop all services if $PRJ_DATA_DIR/procs/ is empy

# Stop services when all registered procs died
while true
do
  sleep 1
  # delete dead procs link
  find $PRJ_DATA_DIR/procs/ -xtype l -delete
  # stop services if folder is empty
  find $PRJ_DATA_DIR/procs/ -type  d -empty -exec stopSvcs \;
done

Possible bug:
$PID is $$ if nix develop
$PID is $PPID of $PPID if is direnv (tested with fish)

[hugosenari]

@zimbatm now that #253 is merged, can we close this?