Weevil

• What is Weevil?
• Key Features
  • Filtering
  • Navigation
  • Analysis
  • Extensible Architecture
• Software Development
  • WPF Application
  • NuGet Packages
• Development
  • Guidelines
  • Compiling
  • Verification
• Recognition
  • Open Source Projects
  • Contributors

What is Weevil?

Weevil is an open-source .NET project that helps analysts extract valuable insights from log files. It's all about boring log files for tasty bytes.

A complete list of features can be found in the release notes.

Key Features

1. File and Record Notes
   • Capture high-level observations as remarks, or low-level details as record comments.
2. Persisted State
   • Automatically load filter history, record comments, and file-level comments when opening a log file.
   • Share the application's state as an XML sidecar with colleagues.
3. Non-Destructive Operations
   • The Weevil application ensures that the original log file is never modified.
4. Simplified Call Stacks
   • When a record includes an exception call stack, Weevil simplifies it by displaying only business logic references.
5. Clear Operations
   • This operation removes records from memory, reducing RAM usage and speeding up filtering.

Filtering

One or more filter criteria can be used to show or hide log file records.

1. Inclusive and Exclusive Filters
   • Display records matching the inclusive filter while hiding those matching the exclusive filter.
2. Filter Criteria
   1. Plain Text
   2. Regular Expressions
   3. Aliases
      • Prefixed with #, these keywords are used to reference built-in complex filters.
      • For example: the #IPv4 alias detects IP addresses using the following regular expression \b(?<IpAddress>(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(?:\.(?:25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})\b
   4. Monikers
      • Prefixed with @, monikers can be used to filter by metadata collected by Weevil.
      • For example, the @Comment can be used to identify records that have a user comment.
3. Multiple Criteria
   • Multiple filter criteria can be combined together using a logical "OR" operator (||).
4. Pinned Records
   • Pinned records are guaranteed to be included in the filter results.

Navigation

1. Find
   • Search for text within filtered results.
2. Go To
   • Jump to specific line numbers or timestamps.
3. Pinned Records
   • Effortlessly navigate between important records.
4. Flagged Records
   • Move between records flagged during prior analysis.
5. Record Comments
   • Navigate between records containing user comments.

Analysis

Use regular expression named groups to identify key data in log files. Then use Weevil's analysis tools to extract data and identify trends.

Each analysis tool updates the Comments field with values that match the provided named group(s) and sets the record's Flagged field.

1. Annotate Records
   • Extract matched named-group values into record comments.
2. First Occurrence
   • Flag the first record for each unique captured value.
3. Last Occurrence
   • Flag the last record for each unique captured value.
4. Stable Value Runs
   • Flag the start and end of repeated value runs.
5. State Transitions
   • Flag when a captured value first appears or changes.
6. Rising Edges
   • Flag when a numeric value increases.
7. Falling Edges
   • Flag when a numeric value decreases.
8. Matching Record Runs
   • Flag runs of consecutive records that match a pattern.
9. Out-of-Order Timestamps
   • Flag records whose timestamps move backward unexpectedly.
10. Measure UI Thread Time
    • Flag records after unusually long UI thread delays.
11. Measure Elapsed Time
    • Calculate the time between consecutive records.
12. Calculate Statistics
    • Calculate summary statistics for selected records.

Furthermore, Weevil supports:

• defining Regions of Interest (ROI)
• creating graphs using named groups

Extensible Architecture

Maximize value by developing domain-specific extensions tailored to your business needs. Weevil can be enhanced with custom plugins:

1. Log File Parsers
   • Create tailored parsers to accurately interpret log files from various sources and formats for seamless integration with Weevil.
2. Log File Analyzers
   • Design specialized analyzers to process and extract valuable insights from the parsed log data, optimizing the analysis for your specific business domain.
3. Dashboard Insights
   • Develop custom dashboard visualizations and insights that highlight the most relevant information, enabling efficient decision-making and a better understanding of your log data.

Software Development

WPF Application

• Installation Guide
• Help Manual

NuGet Packages

Latest Release	NuGet Package
	BlueDotBrigade.Weevil.Common.nupkg
	BlueDotBrigade.Weevil.Core.nupkg
	BlueDotBrigade.Weevil.Windows.nupkg

A .NET application can use Weevil's feature set by directly referencing the BlueDotBrigade.Weevil.Core NuGet package.

For example, you can determine when equipment changed by using the following sample code:

var engine = Engine
   .UsingPath(@"C:\Temp\hardware.log")
   .Open();

// The `UniqueId` regular expression named group is used to
// capture hardware serial numbers.
engine.Filter.Apply(
   FilterType.RegularExpression,
   new FilterCriteria(@"Received hardware message. ID=(?<UniqueId>[a-zA-Z0-9]+)"));

// This type of analysis compares the captured serial numbers
// and flags the record when a value changes.
engine.Analyzer.Analyze(AnalysisType.StateTransitions);

foreach (var record in engine.Filter.Results.Where(r => r.Metadata.IsFlagged == true))
{
   Console.WriteLine(
   $"{record.CreatedAt} {record.Metadata.Comment}");
}

Development

Attribute	Description
	The list of features & bug fixes for the latest Weevil release.
	Source code for the most stable version of Weevil.
	The most up-to-date source code. This branch includes features that are still under development.
	A value of passing indicates that the main branch is compiling and that the automated tests have passed.
	Total size of Weevil's Git repository.
	Indicates when the Git repository was last updated.

Guidelines

• When working on the WPF application, please follow the Style Guide for the user interface.

Compiling

The following steps outline how to build Weevil's WPF application:

1. Download the latest stable release source code.
2. If you have implemented a custom Weevil plugin:
   • Prior to starting Visual Studio, create the following Windows [environment variable][EnvironmentVariable]:
     • %WEEVIL_PLUGINS_PATH% which refers to the directory where the Weevil plugin assembly (*.dll) can be found.
3. Using Visual Studio, compile the WPF project: BlueDotBrigade.Weevil.Gui. [EnvironmentVariable]: https://en.wikipedia.org/wiki/Environment_variable#Windows

Verification

Software integrity is verified through a number of automated tests that can be found in the /Weevil/Tst/ directory:

• UnitTests
• FunctionalTests

Recognition

• Metalama
  • Metalama's aspect-oriented library helps simplify the codebase by reducing boilerplate. Special thanks to the PostSharp Technologies team for creating this successor to PostSharp.
• GitHub
  • Free Git repository hosting for this project and many others.

Open Source Projects

• Live Charts
  • Beto Rodriguez et al. have developed an impressive WPF charting library. We look forward to future releases.
• Material Design in XAML
  • An excellent WPF library that helps standardize themes and improve the overall quality of an application's user interface.
• Cocona
  • Mayuki Sawatari et al. have created an excellent library for building .NET command-line applications.

Contributors

A special thanks to everyone who has contributed to this project.