Reporting tool does not shows services linked to hostgroups

[mrcoffee]

I think this is since my upgrape from RC3 to RC5 last week. See screenshot.

Anyone experiences the same problem?

Merci a vous!

Roderick

[mrcoffee]

Reporting tool shows no services for a host

[jmathis]

I think there are no bugs.

you have to rebuild your reporting

$> /usr/local/centreon/cron/archiveDayLog -r

[mrcoffee]

/usr/local/centreon/cron/archiveDayLog -r

Oh dear, I executed the command and now my complete history of all the host availability information of the last year is gone! And still none of my host have services attached in the reporting tool.
Still no bug? How to solve this?

Salut,
Roderick

[msugumaran]

Hi,
First to reassure you, your last year reporting is not lost :)
The reporting engine has changed. the exec archiveDayLog is run every day to fill the tables "log_archive_host" and "log_archive_service" in Centstorage database. In the last version of the reporting engine, theses tables were filled with the data read in nagios log archives, but now, we read the data in the table log of centstorage. This new method brings significant improvement in the data treatment.
When you executed /usr/local/centreon/cron/archiveDayLog -r, you rebuilt all rows in "log_archive_host" and "log_archive_service" from the table "log" but has the informations are kept only one month in this table, you get the reporting only for the last month.
There is an intermediate step to get the reporting since the first run of nagios.
Run : /usr/local/centreon/bin/logAnalyser -a
This will get all nagios log archives and insert them in Centstorage "log" table. Then launch "archiveDayLog -r", you will get back all your data in the reporting.

About the reporting of your services, it's not normal that you get nothing. Could you attach your centstorage database tables structure? Maybe some tables haven't been updated during your upgrade of centreon.

Finally there is still one bug that will be corrected in the next release. The reporting works, but some services have 100% of undetermined time even if they are checked by nagios. We are working on it, so you will soon have a reporting tool completly stable.

Of course we should have precise the different steps in the centreon web installer to re-integrate correctly the reporting tool. That's our mistake :)
Sorry for this.

I'm waiting for your reply to analyse your centstorage DB structure.
Regards,
MaT.

Replying to mrcoffee:

/usr/local/centreon/cron/archiveDayLog -r

Oh dear, I executed the command and now my complete history of all the host availability information of the last year is gone! And still none of my host have services attached in the reporting tool.
Still no bug? How to solve this?

Salut,
Roderick

[mrcoffee]

Thnx for you reply.
The command started to work with files that are 30 days old, not with older files althought they are there over 400 files in the directory /usr/local/nagios/var/archives

# /usr/local/centreon/cron/archiveDayLog -r
/usr/local/nagios/var/archives/nagios-09-03-2008-00.log
/usr/local/nagios/var/archives/nagios-09-04-2008-00.log
/usr/local/nagios/var/archives/nagios-09-05-2008-00.log
/usr/local/nagios/var/archives/nagios-09-06-2008-00.log
/usr/local/nagios/var/archives/nagios-09-07-2008-00.log
/usr/local/nagios/var/archives/nagios-09-08-2008-00.log
/usr/local/nagios/var/archives/nagios-09-09-2008-00.log
Use of uninitialized value in substitution (s///) at /usr/local/centreon/bin/logAnalyser line 107, <FILE> line 2185.
Use of uninitialized value in concatenation (.) or string at /usr/local/centreon
/bin/logAnalyser line 125, <FILE> line 2347.
... (a lot of errors on this file but that's something different i guess)
...
/usr/local/nagios/var/archives/nagios-09-10-2008-00.log
/usr/local/nagios/var/archives/nagios-09-11-2008-00.log
/usr/local/nagios/var/archives/nagios-09-12-2008-00.log
/usr/local/nagios/var/archives/nagios-09-13-2008-00.log
/usr/local/nagios/var/archives/nagios-09-14-2008-00.log
/usr/local/nagios/var/archives/nagios-09-15-2008-00.log
/usr/local/nagios/var/archives/nagios-09-16-2008-00.log
/usr/local/nagios/var/archives/nagios-09-17-2008-00.log
/usr/local/nagios/var/archives/nagios-09-18-2008-00.log
/usr/local/nagios/var/archives/nagios-09-19-2008-00.log
/usr/local/nagios/var/archives/nagios-09-20-2008-00.log
/usr/local/nagios/var/archives/nagios-09-21-2008-00.log
/usr/local/nagios/var/archives/nagios-09-22-2008-00.log
/usr/local/nagios/var/archives/nagios-09-23-2008-00.log
/usr/local/nagios/var/archives/nagios-09-24-2008-00.log
/usr/local/nagios/var/archives/nagios-09-25-2008-00.log
/usr/local/nagios/var/archives/nagios-09-26-2008-00.log
/usr/local/nagios/var/archives/nagios-09-27-2008-00.log
/usr/local/nagios/var/archives/nagios-09-28-2008-00.log
/usr/local/nagios/var/archives/nagios-09-29-2008-00.log
/usr/local/nagios/var/archives/nagios-09-30-2008-00.log
/usr/local/nagios/var/archives/nagios-10-01-2008-00.log
/usr/local/nagios/var/archives/nagios-10-02-2008-00.log
/usr/local/nagios/var/archives/nagios-10-03-2008-00.log

# /usr/local/centreon/bin/logAnalyser -r

Result: history in Centreon is still 30 days in the reporting tool.

In my centreon all services are deleted from the webinterface. I just found one service that I created two weeks ago, but it does not contain any data, while the checks are OK. I added a screenshot.

I will also try to give you what you needed concerning the database tables. i'm not a mysql admin (yet) so if it's not correct, please give me a hint.

Hope to hear from you!

Roderick

+----------------------------------------+
| Tables_in_centreon |
+----------------------------------------+
| acl_group_contacts_relations |
| acl_group_topology_relations |
| acl_groups |
| acl_res_group_relations |
| acl_resources |
| acl_resources_hg_relations |
| acl_resources_host_relations |
| acl_resources_hostex_relations |
| acl_resources_sc_relations |
| acl_resources_service_relations |
| acl_resources_sg_relations |
| acl_topology |
| acl_topology_relations |
| cfg_cgi |
| cfg_nagios |
| cfg_ndo2db |
| cfg_ndomod |
| cfg_resource |
| command |
| command_categories |
| command_categories_relation |
| contact |
| contact_host_relation |
| contact_hostcommands_relation |
| contact_param |
| contact_service_relation |
| contact_servicecommands_relation |
| contactgroup |
| contactgroup_contact_relation |
| contactgroup_host_relation |
| contactgroup_hostgroup_relation |
| contactgroup_service_relation |
| contactgroup_servicegroup_relation |
| cron_operation |
| css_color_menu |
| dependency |
| dependency_hostChild_relation |
| dependency_hostParent_relation |
| dependency_hostgroupChild_relation |
| dependency_hostgroupParent_relation |
| dependency_metaserviceChild_relation |
| dependency_metaserviceParent_relation |
| dependency_serviceChild_relation |
| dependency_serviceParent_relation |
| dependency_servicegroupChild_relation |
| dependency_servicegroupParent_relation |
| escalation |
| escalation_contactgroup_relation |
| escalation_host_relation |
| escalation_hostgroup_relation |
| escalation_meta_service_relation |
| escalation_service_relation |
| escalation_servicegroup_relation |
| extended_host_information |
| extended_service_information |
| general_opt |
| giv_components_template |
| giv_graphs_template |
| host |
| host_hostparent_relation |
| host_service_relation |
| host_template_relation |
| hostgroup |
| hostgroup_relation |
| informations |
| meta_contactgroup_relation |
| meta_service |
| meta_service_relation |
| modules_informations |
| nagios_macro |
| nagios_server |
| ns_host_relation |
| ods_view_details |
| on_demand_macro_host |
| on_demand_macro_service |
| options |
| service |
| service_categories |
| service_categories_relation |
| servicegroup |
| servicegroup_relation |
| session |
| timeperiod |
| topology |
| topology_JS |
| traps |
| traps_service_relation |
| traps_vendor |
| view_img |
| view_img_dir |
| view_img_dir_relation |
+----------------------------------------+
91 rows in set (0.00 sec)

[mrcoffee]

Excusez moi, I mixed up the two commands in the text above, not on the server ;)
# /usr/local/centreon/bin/logAnalyser -a
et
# /usr/local/centreon/cron/archiveDayLog -r
But i think you figured that out as you can see in the output.

And I found one other host (of over 200) that had its services attached in the reporting tool, this time with status data of the last 30 days.

Merci

[mrcoffee]

added a mysqldump of the centreon database

[msugumaran]

There are two databases with centreon :
-centreon
-centstorage
I need centstorage DB structure (and it's content, if it's not too big)

[mrcoffee]

Offcourse. It's almost 70Mb so I dont know it I can upload it. i can pput it on webserver tonight.

mysql> show tables from centstorage;
+-------------------------+
| Tables_in_centstorage |
+-------------------------+
| config |
| data_bin |
| data_stats_daily |
| data_stats_monthly |
| data_stats_yearly |
| index_data |
| instance |
| log |
| log_archive_host |
| log_archive_last_status |
| log_archive_service |
| log_snmptt |
| metrics |
| statistics |
+-------------------------+
14 rows in set (0.00 sec)

[mrcoffee]

Upload to the trac.centreon is maximized to a filesize lower then 70MB. Here is the URL where you can download the database:
http://www.r71.nl/centstorage.gz

[mrcoffee]

-bash-3.2# mysqldump --no-data -u root -p centstorage
Enter password:
-- MySQL dump 10.11
--
-- Host: localhost Database: centstorage
-- ------------------------------------------------------
-- Server version 5.0.45
...

and so on -> see the attached file centstorage_db_schema.txt

[mrcoffee]

Bonjour,
As tu eu l'occasion de travailler sur ce problème?
Si tu veux plus de information, dit moi svp.

Merci!

Roderick

[jmathis]

Hi,

Can you test with : r6808.

thanks

[mrcoffee]

Thnx for tyour reply. Downloaded the file and placed in in the same directory as the original file. Then tried to run it but it fails:

-bash-3.2# /usr/local/centreon/cron/archiveDayLog -r
Possible unintended interpolation of @CENTREON_ETC in string at /usr/local/centreon/cron/archiveDayLog line 46.
Global symbol "@CENTREON_ETC" requires explicit package name at /usr/local/centreon/cron/archiveDayLog line 46.
Execution of /usr/local/centreon/cron/archiveDayLog aborted due to compilation errors.
-bash-3.2# /usr/local/centreon/cron/archiveDayLog -r -d
Possible unintended interpolation of @CENTREON_ETC in string at /usr/local/centreon/cron/archiveDayLog line 46.
Global symbol "@CENTREON_ETC" requires explicit package name at /usr/local/centreon/cron/archiveDayLog line 46.
Execution of /usr/local/centreon/cron/archiveDayLog aborted due to compilation errors.

[mrcoffee]

Ok, I changed the script:
#require "@CENTREON_ETC@/conf.pm";
require "/etc/centreon/conf.pm";

Now I could run it.

But still no result:
- history does not return, got less history because of importing just the last month
- services do not show up in the reporting tool

-bash-3.2# /usr/local/centreon/bin/logAnalyser -a
/usr/local/nagios/var/archives/nagios-09-15-2008-00.log
/usr/local/nagios/var/archives/nagios-09-16-2008-00.log
/usr/local/nagios/var/archives/nagios-09-17-2008-00.log
...
/usr/local/nagios/var/archives/nagios-10-15-2008-00.log
-bash-3.2# /usr/local/centreon/cron/archiveDayLog -r

[msugumaran]

Could you upgrade to Centreon.2.0-RC6 and retry the re-building of the reporting statistics :
-bash-3.2# /usr/local/centreon/bin/logAnalyser -a
-bash-3.2# /usr/local/centreon/cron/archiveDayLog -r
Is it working fine?
Normally the last version fixes all bugs on the reporting tool

[mrcoffee]

I ugraded to RC7, it was released today. And I experience the same problems.

- /usr/local/centreon/bin/logAnalyser -a only imports the last month, houw can I import all?

- /usr/local/centreon/cron/archiveDayLog -r finishes without a problem

- None of the 1013 services are visible in the reporting tool except 4 or 5 services.

So... do you think you have a solution for me?
Or do I have to install everything from scratch and use sqldump to export the database and import it in the new setup again?
Or do I have to reinstall one of the databases again? Centreon Database holds the configuration doesn't it? I'd like to keep the performance data but I can start over again, import the 'Centreon Database' (Centreon configuration) in an new setup and also import nagios history again.

Qu'est que vous pensez?

[msugumaran]

I couldn't download your centstorage database archive. I would like to do a final test before you restart the installation of Centreon. Could you send me all the files contained in the directory "var/archives/" of nagios

[msugumaran]

And I will also need the content of the database centstorage (for the last time :) )without the content of the table data_bin which must be the biggest table of centstorage.

[mrcoffee]

Ok, merci. I just collected the data and you can download it.
- http://www.r71.nl/mysql_centstorage_db.gz (90MB)
- http://www.r71.nl/nagios-archives.tar.gz

(I don't receive any email when you udate thise ticket, is that normal behavior?)

[mrcoffee]

centstorage: table data_bin without data

[mrcoffee]

I didnt see your remark about not dumping the data in the table no_data. You can download the files:

no_data table without data:
http://trac.centreon.com/attachment/ticket/249/mysql_centstorage_db_nodata_table_data_bin.gz

all tables except no_data:
http://www.r71.nl/mysql_centstorage_db_all_tables_except_no_data.gz

[msugumaran]

I've tested your centreon database with a pre-installed centreon. All the services that you have defined in Configuration > services > services by hosts appear in the reporting. But all the services that you have defined by hostgroup cannot be viewed in the reporting. Maybe it's your real problem. It's not a bug but rather a missing feature. Actually, Services by hostgroup are not taken in account. We will fix this in the futur releases.
Tell me if it answers to your problem.

[mrcoffee]

Merci pour votre message.
C'est étrange parce que dans RC1 to RC3 I did not have this problem, it started since the upgrade from RC3 to RC5.

-Does this mean I'm using a non-standard way to connect services to hosts by using groups?
- If I connect the service directly to the host, will this solve the problem?
- With the future release, do you mean RC8 or Centreon 2.1?

[msugumaran]

You can use services linked with hostgroup. In the RC5, we have conpletly changed the reporting engine, because the previous one could not evolve properly. And we have to optimize the computing times. That's why you lost this feature. But it will be soon re-integrated for centreon 2.0 stabble version.
And if you want a standart way to configure your monitoring. Use hosts templates instead of hostgroups and use relations between hosts templates and services templates to generate automatically services linked to hosts.

[mrcoffee]

Thank you for your clear explanation. I'll set up our system in the way you explained for new hosts and if possibel for current hosts. And I'll be looking forward to the new RC version.

One thing left: how can I import the old data from the Nagios archives?

[jmathis]

juste run archiveDayLog like this :

./archiveDayLog -r

[mrcoffee]

Nope, pas un succes. And for some reason the data for today is gone now too.

See the screenshot: http://trac.centreon.com/attachment/ticket/249/20081027_centreon_reporting_problem.JPG

-bash-3.2# pwd
/usr/local/centreon/cron
-bash-3.2# ./archiveDayLog -r

[mrcoffee]

FYI:
Le gateaux (the "pie") and the timeline of yesterday is 100% again. The timeline of today is still empty. And history goes back just 30 days.

[mrcoffee]

I hope you don't mind me asking if you can help me with the import again of the old data from the Nagios archives, or do I have to solve this in another way?

Merci,
Roderick

[msugumaran]

We have removed the reporting on the current day. We are estimating that the reporting must be based only on past days.
Now services linked to hostgroups can be viewed in the reporting : r7037

[mrcoffee]

Merci,
I was talking about the import of my Nagios archives that go back to oktober 2007.
I lost this when we were troubleshooting in the beginning of this ticket.

Any suggestions?