Setting Up distributed environments

Support requests, bug reports, questions etc.
robertkoeppl
NConf rookie
NConf rookie
Posts: 19
Joined: Mon Mar 05, 2012 11:01
Location: Leoben Austria

Setting Up distributed environments

Post by robertkoeppl » Mon Mar 05, 2012 11:23

Hi!
I am having trouble setting uop a distributed environment.
I have numeroust sites with a local nagios server and a central server.
I have added a remote server as collector als added some of the hosts there. I set them to be monitored by that remote server and they are.
However they do not show up in the configuration of the central server at all.
Could you please point me in the right direction on how to fix that.
Possibly related: adding advanced services fails with the central monitoring server spitting errors that the host name field would be empty, the remote configuration is created without issues.
The central server also needs to monitor some hosts directly - which are added without any issues.
Because of the infrastructure we have it is only possible to "pull" data, so I intend to write the stati to file and import them with log2ido locally. However that still needs the hosts and checks to exist in the central configuration as they would otherwise be ignored.

robertkoeppl
NConf rookie
NConf rookie
Posts: 19
Joined: Mon Mar 05, 2012 11:01
Location: Leoben Austria

Re: Setting Up distributed environments

Post by robertkoeppl » Mon Mar 05, 2012 12:01

Seems this is related to this:
viewtopic.php?f=8&t=405
I found an explanation and a workaround for the "missing" hosts:
as there can still be no monitor with the same name as a collector I had created a different name there. Thus I had to modify my deploy scripts to includer that folder, copy it ot a subfolder if the icinga/nagios config and include that folder in the config.
regarding the advanced services it seems that I would need to add at least one host monitored by the central machine (in its role as collector) to be able to use this.
This does not make too much sense to me.
Nconf sees this host as a collector as any other as well. I can think of numerous scenarios, where an advanced service is needed on a hostgroup monitored by one collector and not needed anywhere else.

User avatar
scrat
NConf enthusiast
NConf enthusiast
Posts: 101
Joined: Mon Aug 23, 2010 16:28
Location: Vienna, Austria
Contact:

Re: Setting Up distributed environments

Post by scrat » Mon Mar 05, 2012 17:15

Hi,

I strongly suggest that your central monitor does NOT monitor any hosts.
It's possible to have a setup with collectors and an activly monitoring monitor, but this is horrible if your setup grows.
So, better don't let your monitor check any hosts.

The simplest setup for a distributed environment is:
- central monitor which receives passive checks
- one-multiple active/passive checking collectors which are sending their check results to the monitor (e.g. via nsca)

All hosts and services (also advanced services) are assigned to the collector which is choosen in filed "monitored by" and to the monitor (you have to define on in the monitor-section of NConf).

If you have further questions, let me know.
I've a distributed setup running with NConf 1.3 and Icinga+nsca,...


Regards,
René

robertkoeppl
NConf rookie
NConf rookie
Posts: 19
Joined: Mon Mar 05, 2012 11:01
Location: Leoben Austria

Re: Setting Up distributed environments

Post by robertkoeppl » Mon Mar 05, 2012 17:33

NSCA is a NoGo because of the requirement that the remote sites has no active connection back to main site (not allowed for various reasons). Central monitor needs to collect as there is no hardware budgeted for a collector on this site. Possible solution would be to have a nagios running, that reports the data to the icinga on the same host.
Tha status on the central host is allowed to lag a bit, as it is mainly used for reporting purposes. currently we use remote Installations that write files that are fetched by rsync onca per day, but that is rather clumsy and not really possible with a database backend running on the remote site (needed for the local reporting and nagvis running there).
If I understood the IDO and Icinga Documentation correctly it is not possible to write to DB and to file at the same time. otherwise I could just fetch that files every five minutes and import them using log2ido.
Regards, Robert

robertkoeppl
NConf rookie
NConf rookie
Posts: 19
Joined: Mon Mar 05, 2012 11:01
Location: Leoben Austria

Re: Setting Up distributed environments

Post by robertkoeppl » Thu Mar 08, 2012 09:25

It seems I got a solution now - allthough a bit complex.
I have created a local clone of my icings installation and renamed the startscript, folder and binary accordingly. This copy does the active checking that has to be done form the local installation. it submits its checks via nsca to the main instance of icinga running on the same server which handles the webinterface and the database backend.
The remote installations write to a local database, as there is some reporting going on on each site, but also obsess over the checks and write the results to a file. that file is actively collected from the central site via rsync run by cron.
It delets the data offsite as soon as it has been transmitted to the central site and imports it by sending the file to the NSCA. This gives a slight delay of little more than a minute to the ,local updates, but that is acceptable as the alerting is handled by the local installations.
The deploy script needed some adjustments for that, but works fine now.

Locked