Kleene: A commonality matcher for large text files

Kleene generates patterns composed from a group of log files that share common data. It can then use this data to then match against a larger set of log files to determine which of them matches the content.

Soon, you will be able to register your logs in a database for ongoing pattern revision. Additionally, further enhancements to the pattern generation algorithm are expected to come soon.

Installation

Get a copy of cargo, a tool to drive rust builds. Then,

cargo install --git https://code.hollensbe.org/erikh/kleene

Usage

There are currently two commands:

• kleene train <files> > pattern_file, which emits a pattern over standard output; kleene patterns have null characters in them (\0) so it is important they are not copy-pasted. Redirect standard output to a file.
• kleene match <files> < pattern_file accepts a pattern over standard input and matches it against the files, terminating immediately on the first mismatch currently.

It should perform reasonably well on large files.

Author

Erik Hollensbe github@hollensbe.org

License

MIT