Adventures of a Small Time OpenStack Sysadmin Chapter 034 - Central Logging

Adventures of a Small Time OpenStack Sysadmin relate the experience of converting a small VMware cluster into two small OpenStack clusters, and the adventures and friends I made along the way.

Adventures of a Small Time OpenStack Sysadmin Chapter 034 - Central Logging

"Central Logging" is what Kolla-Ansible calls a Monasca Project installation plus some additional software to do generic logging stuff.

There are tasks to perform upon first login as seen at

https://docs.openstack.org/kolla-ansible/yoga/reference/logging-and-monitoring/central-logging-guide.html

Because monasca_log_persister refuses to connect to new version of elasticsearch, central logging is dead.  I opened a bug on the problem.  It seems Monasca as a project is completely dead (despite very excited YouTube videos about it replacing Ceilometer a couple years ago) and it only works with a very old version of elasticsearch from some years ago.

This bug also means a lot of resource use on the controller as "monasca_log_persister" container will restart repeatedly every couple seconds trying to connect to a valid elasticsearch, so it needs to be manually killed off at the command line or it'll flood the system with error messages about its bootlooping restarting over and over forever.

https://bugs.launchpad.net/kolla-ansible/+bug/1980554

If central logging ever works again, Elasticsearch Curator needs some configuring later to prevent flooding the logging disk, but we'll cross that bridge when we come to it.

Also there is a show-stopper bug where trying to click on the "monitoring" "overview" page, which would presumably be the main page for Monasca, simply crashes Horizon, as I read the bug report, something changed in Horizon some years ago, and Monasca being a dead project its never going to be updated to be compatible.

Its a short, depressing update post, but that's how things are, sometimes.  An integrated monitoring system for OpenStack would be nice.  Until then, I just watch metrics and graphs on a Zabbix system that is not integrated with OS but works pretty well.

Tomorrow, struggle, and eventual complete success, with Neutron.

Stay tuned for the next chapter!