Async to remove old log file

[SpringHgui]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. For example, "I'd like to do x but currently I can't because y [...]".

1. At the beginning of the project, my configuration was like this，72 log files will be retained

 configuration
    .WriteTo.File($"Logs/log-.log",
        rollingInterval: RollingInterval.Day,
        retainedFileCountLimit: 72 ,
        shared: false,
        outputTemplate: temp);

2. Since the log file is relatively large, compression is used for archiving.
   my configuration was like this，2 log files will be retained and 70 compressed log files， ArchiveHooks

 configuration
    .WriteTo.File($"Logs/log-.log",
        rollingInterval: RollingInterval.Day,
        retainedFileCountLimit: 2,
        shared: false, hooks: new ArchiveHooks(70),
        outputTemplate: temp);

serilog-sinks-file/src/Serilog.Sinks.File/Sinks/File/RollingFileSink.cs

Line 206 in 2a61b4b

_hooks?.OnFileDeleting(fullPath);

3. Everything looked fine, but when I upgraded the program to a production environment, the program blocked on startup. Unable to complete startup after several minutes
4. The first startup after the program upgrade requires compressing 70 files, which takes a lot of time. The program will be blocked until this deleting work is completed

Describe the solution you'd like

Archive files are usually compressed or copied, which takes a lot of time. During this period, the main program should not be blocked from starting. Async to remove old log file supported

serilog-sinks-file/test/Serilog.Sinks.File.Tests/Support/ArchiveOldLogsHook.cs

Line 20 in 2a61b4b

System.IO.File.Copy(path, newFile, false);

Describe alternatives you've considered
A clear and concise description of any workarounds or alternative solutions you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[cocowalla]

You mentioned ArchiveHooks - are you using this Serilog plugin? If so, this issue should be opened in that repo, rather than this one.

If I understand correctly, this is a one-time issue the first time you start the app after adding in compression via ArchiveHooks - do you think such a rare event warrants changes? The serilog-sinks-file-archive plugin relies on the (synchronous) OnFileDeleting hook - it has to do the work there (compression), then return control to the main Serilog File sink which will continue to delete the file; I'm unsure how we could prevent blocking.

[SpringHgui]

@cocowalla I looked at the source code, serilog-sinks-file-archive can't be asynchronous, it must be supported by serilog-sinks-file

[cocowalla]

@SpringHgui that's kind of what I was getting at 😄

But for the issue to be actionable here, might I suggest you couch it in more general terms (rather than related to an issue with one specific plugin), provide a justification, and possibly provide the bones of a solution?

[SpringHgui]

serilog-sinks-file/src/Serilog.Sinks.File/Sinks/File/RollingFileSink.cs

Line 173 in 2a61b4b

ApplyRetentionPolicy(path, now);

change to this?

  Task.Run(() =>
  {
      ApplyRetentionPolicy(path, now);
  });

Increase the speed of program startup and asynchronously archive expiration log file

[nblumhardt]

The retention process there is performed under the _syncRoot mutex, so just kicking the operation off in a Task won't be sufficient - either the body of the task would need to take the lock (also blocking writes on the main thread) or we'd hit data races.

Perhaps another option @SpringHgui would be to rename the file inside the archive hook, e.g. to {oldFilename}-archiving.tmp, and kick off an async task to perform the compression there?

In theory you could achieve this by wrapping the lifecycle hook provided by Serilog.Sinks.File.Archive 🤔

Hope this helps,
Nick

[yoav-melamed]

Hi,
I wrote an 'AsyncFileSink' that includes the following features:

• Embedded asynchronous log writing using the Channel<T> API
• Control over the channel capacity
• Advanced rolling policies
  • Rolling by file size
  • Rolling on startup
  • Roll to archive folder
  • Delete old rolled files by age

You can log an issue in my repo for more features and requests

Would be happy to get your feedback:
https://github.com/yoav-melamed/serilog-sinks-asyncfile/