Let’s be honest, debugging error messages generated by VMware Big Data Extensions (BDE) can be painstaking, tedious and tiresome. Having recently begun to rely on VMware Log Insight more and more, I determined the deployment of BDE v2.3.1 would leverage the Log Insight agent. During the actual vApp deployment of BDE, you may have noticed an option to specify a remote syslog server. I chose to leave the option blank during the deployment, instead choosing to install and configure the Log Insight agent post-installation.
The Linux Log Insight agent can be downloaded on the Administration->Agents screen at the very bottom of the screen.
The download link will include the IP address of the Log Insight server and is used during the RPM deployment of the agent.
Management Server Installation
The BDE management server is currently running CentOS 6.7. After copying the agent to the management server, the following commands can be executed to install and perform the service configuration.
# rpm -ihv VMware-Log-Insight-Agent-3.0.0-2985111.noarch_192.168.1.2.rpm # chkconfig liagentd --list # chkconfig liagentd on # service liagentd restart
After the installation is complete, the BDE management server should appear in the list of servers with installed agents.
The next thing to do is begin editing the /etc/liagent.ini configuration file to send the Serengeti log files to the Log Insight server. Mr. Steve Flanders has an article from 2014 that describes the process of adding custom log files for the agent to parse.
41 [filelog|syslog]
42 directory=/opt/serengeti/logs
43 include=serengeti.log;ironfan.log
44 event_marker=\[\d{4}-\d{2}-\d{2}T
BDE Template Installation
Having the Log Insight agent installed on the BDE management server is helpful, but I determined having the agent installed natively during the all deployments of Hadoop and Apache Mesos clusters would be even more helpful. There were two options for installing the agent during deployments:
- Create a Chef cookbook and include the Log Insight agent RPM file on the management server repo.
- Install and configure the agent on the template VM itself.
I opted for option #2, merely because it would be the quickest way initially. Admittedly, using a Chef recipe would have been the better long-term and more “DevOps-y” way to perform the installation. I may reconsider my choice in the future.
Just like the management server installation process, copy the Log Insight agent RPM file onto the template and install it using the same steps. No modifications to the /etc/liagent.ini file are necessary for the VM template node because the Serengeti logs don’t exist.
Be sure to delete the snapshot on the VM template node so the changes take effect.
Now all of the VMs deployed through VMware Big Data Extensions will immediately send log updates to Log Insight and the logs for all of the deployments are now captured as well. Having the logs accessible through Log Insight let you parse them with the power of the Log Insight UI and the filtering capabilities there.
Enjoy.
