Nova-Agent : How it Works on OpenStack Public Cloud

This is brief introduction to Nova-Agent, which runs as daemon on OpenStack Public Cloud like Rackspace. If it does not start, dependent services will fail. In previous article, we talked about fixing patching ShellShock Bug, but we have not said that Xen Virtualization had some bugs too, which was so sensitive, globally all including Rackspace Patched and Restarted the Servers. With one level up patching, it is not unlikely to miss nova-agent up and running rightly, or it might get missing after manual upgrade (with apt-get upgrade command) and accepting to replace a particular file. By default we never replace it to avoid this issue on PVHVM instances. This is an article which will partially explain the function of Nova-Agent and How it Works on OpenStack Public Cloud. Without understanding Linux from Kernel level, OpenStack Nova, you should not play on production server.

Nova-Agent : How it Works on OpenStack Public Cloud And What Will Happen If It Is Not Running

Your server will run like an isolated instance with no API based access to other services, usually the dependent services run but throw error. Services include Cloud Monitoring Tool, Image Backup etc. When we create an instance, automatically this nova-agent gets ready. What is untold, you can stop it incase you want to use the server for simple reason. Because, as it is a daemon, with restart, it will restart and suck some memory. If you run this command, you’ll understand it is there :

Unfortunately, service nova-agent restart or service nova-agent start will never give you anything worthy. If you do not get any output, the silence will say “I am running but in wrong path”, if get this error “‘import site’ failed; use -v for traceback”, frankly you have been screwed, take file level backup, backup of folders like /etc, because practically on any unix like OS, putting configuration file or other stuffs works fine, provided that, the servers are of same version, same thing. That is why you can run Application on Mac without really dragging to Application folder and that is the primitive cheat to use the paid application forever. The Application gets mounted. So, if you create a storage device, mount it on cloud server and copy all the stuffs, it will work fine as backup. You will unmount it, then troubleshoot. Victor wrote a good guide to easily troubleshoot :

xe-linux-distribution service is provided by xe-guest-utilities package, this runs on all Linux servers on Rackspace Cloud. This is the service which is responsible for communicating Virtual Machine (your device) with the hypervisor (the hardware). How much important this xe-linux-distribution? You can not change the password from Rackspace control panel, you can not use the Rackspace console. xe-linux-distribution must start before the nova-agent service. The upgrade with accepting the fix actually fully FCUKS this order.

It is a huge pain and complicated process to re-install it, that is why Rackspace always provide 100% managed support for Nova-Agent related issues. They will not login to the device in case of Infrastructure Managed instance. I personally think, Rackspace should add a paid add-on service for some works on device by them. Otherwise a noob might hire outsider which is more dangerous for the total environment.

Nova-Agent : How To Fix If It Is Not Running

Run this command :

If you have received kind of this error as output :

You should thank God. Do this :

It will show you a Python script and a version-numbered folder – like 1.39.0. CD there :

There is a Plugins folder, cd to that folder. The advantage of Open Source is that, you’ll get it here :

First make a directory and move the existing files in that. You can use FTP if you want. Next wget the sources from there in Github. Replacing the problem creating files might fix your condition, but this can unfortunately open the ShellShock vulnerability. If the problem is with XenStore, the only way is to reinstall the xe-guest-utilities. It is difficult and honestly Rackspace make it an exception to login to the device and fix it. I myself can not take the risk to do it – it is very difficult, if one step goes wrong, the server which was running, it will die. Citrix has peculiar stuffs.

So, what you will do in case Rackspace’s official on chat help can not make you to execute the things. You should create similar instances and simply build it again, before deleting old instance, rebuild it from original image (to preserve the IP), again re-configure the web server or application server. Although this is painful, this is easier for even the beginner. You kept the files of /etc/ like folder, from it you have to understand what configuration your Nginx server had. It might turn towards good for you.