Why nxlog?

[drnic]

I was looking at https://github.com/hybris/nxlog-boshrelease and now this boshrelease and both use nxlog - why do you like it?

My dislike starts with a bug/feature - I don't seem to be able to specify a glob or globs for input files (your own workaround is hefty [1])

But then I go looking for the source code and there is no github/whatever repo. It's a private project that ships a community edition and an aged "Source code" download. [2]

What are the pluses for nxlog?

[1] https://github.com/logsearch/logsearch-shipper-boshrelease/blob/master/jobs/logsearch-shipper/templates/bin/logs.generate-config.erb
[2] https://nxlog.org/products/nxlog-community-edition/download

[dpb587]

I'm not personally very tied to it - nxlog was something we were
initially working with for clients shipping to logsearch deployments. Nxlog
was handling some networking and locking issues in Windows better than
others we tested. Since nxlog could be compiled for many platforms, decided
to get started with it here. Since then though, they haven't provided the
latest sources so it has become more of a liability. Other affirmations for
using it in this project were - the config was easy to learn, it was easy
to debug, it was trivial to use arbitrary fields/tags for separate log
files, it used minimal resources. I wasn't aware of the other nxlog bosh
release when this started, if it existed.

I mentioned earlier we were thinking it'd be nice to switch away from nxlog
and just use the stemcell-included rsyslog. Assuming it can be as flexible,
the job properties shouldn't need to change, just generate-config. I've
just been preoccupied elsewhere and it's a non-trivial change.

If globs aren't working, it's a bug. If you'd like to post your snippet I
can help figure out why. We've used globs and haven't had an issue,
assuming it's a glob format Ruby's Dir.glob can handle. The other thing to
note is that because we enumerate the file system and then write out the
nxlog config, the log files must already exist which is typically assumed
to happen when monit services start up. You can use the
logsearch.logs.start_delay to further delay this (defaults to 60 seconds)
or avoid using globs.

This configuration for this approach isn't optimal, so it's been sort of a
compromise to meet the desired configuration features - namely, bosh
releases can define their log files, deployment manifests can, or both. The
complicated config-generating globbing is supposed to merge all those
customized log file fields/tags appropriately (vs a typical log retailer
which can only have a single directive for a glob path and a single set of
extra fields/tags in it). If that sort of makes sense.

Happy to discuss/clarify/brainstorm further.

On Friday, November 20, 2015, Dr Nic Williams notifications@github.com
wrote:

I was looking at https://github.com/hybris/nxlog-boshrelease and now this
boshrelease and both use nxlog - why do you like it?

My dislike starts with a bug/feature - I don't seem to be able to specify
a glob or globs for input files (your own workaround is hefty [1])

But then I go looking for the source code and there is no github/whatever
repo. It's a private project that ships a community edition and an aged
"Source code" download.

What are the pluses for nxlog?

[1]
https://github.com/logsearch/logsearch-shipper-boshrelease/blob/master/jobs/logsearch-shipper/templates/bin/logs.generate-config.erb

—
Reply to this email directly or view it on GitHub
#11.

Danny Berger
http://dpb587.me