In this final part of the series, let us quickly recap the things we discussed in Part 1 and Part 2 of this series. We discussed about basics of how software development cycle goes in real, bsics of types of virtualization, basic reason to think about using virtualization in practice, we discussed about lesser know practical matters behind real life software development of inside any company, power consumption, cost of hardware, difficulty in procuring hardware parts are not what commonly known matters. Here is the Third and Final Part of Virtualization Requirements for DevOps.
Virtualization Requirements for DevOps
Deployment chain
Deployment Chain describes the entire process between the completion of a piece of code by the developer and the live operation. Deployment chains are largely automated, but may also contain interruptions for human inspection. Often they also act differently, depending on the requirements and target system. A deployment chain is a complex set of tasks. Since the environment is often very heterogeneous, either historically or due to requirements, deployment chains consist of diverse systems that are strung together. So it happens that they do not always fulfill their purpose according to the DevOps idea perfectly. Since the DevOps idea is very new, there are still no suitable for every situation and every requirement appropriate overall solution. Deployment chains often include the three areas of Continuous Integration,
---
They also adopt the test strategy/strategies. It is thus possible to check fully automatically whether a code complies with the guidelines (such as style guides) or whether error sources (frequent exceptions) can be identified. Unit tests can also be used to ensure that no other code that may be built up on the work site has been accidentally damaged or has undesirably altered its behavior.
From the finished, and as far as possible technically tested, code, a VM or container is created, which must be approved by the QA department and/or the customer. Ideally, the employee can start or run the VM himself without manual assistance from an admin or IT department. It is also conceivable that in the case of a website, the VM or the container automatically starts and the QA department is provided a corresponding link.
In addition, automation can save time by eliminating repetitive work, as well as assembling a VM, by one person. In addition, the deployment chain should include a feedback feature to let developers know right away that there are problems, whether it has detected a deeper error in the automated tests or QA.
Continuous delivery goes a bit further. There even the QA department is automated. After all automatic tests have been successfully completed and the technical functionality has been confirmed, the functionality is loaded directly onto the production system via Continuous Delivery and then monitored using A/B tests. After a short time it is decided whether the changes are in order or have to be changed. However, for this to be possible, there must be a large user base so that the small number of (ignorant) testers is representative. Similarly, the update steps must be very small and always designed for older code in the ground. For example, you can not publish changes to a database schema or the internal structure of the software in this way. Mostly CI and continuous delivery are used, but any continuous deployment may be used.
Realistic Test Systems
The program code (especially scripting languages) often behaves differently on different systems. In addition, servers may have different operating systems, software versions, and libraries. Since the VMs are at least identical to each other in software, this problem is solved. Ideally, we can even use the same VM in live operation. Of course, it needs to be reconfigured and adjusted if necessary (more memory), but the installed software remains the same. In order to always be up to date you should adapt the development environment as far as possible to the production environment. Among other things, one can use a clone of the live database. The goal is to keep differences as small as possible. It also makes it easier to readjust problems
Another important point is that you can build virtual networks and perform load tests there. If there is malfunction in the code, or the VM crash, that is not a problem, since the live system remains untouched. Without a protective layer (achieved by virtualization) it could lead to a complete system crash.
Using loadbalancer, it is also possible to move slowly to the new system. This way, new connections are directed to the upgraded system, but existing connections to the old system are not interrupted. The users do not notice anything about system updates.
Service virtualization is similar to the development of an aircraft. Engineers can not wait to fully assemble an aircraft with their test, instead they model the components in a computer simulation and test them within a smaller, simulated, and safe environment. The individual components correspond to the parts of the flight during software development. Faster delivery times, flexible customization, and early detection of vulnerabilities make it clear how software development processes can be improved.
The productive operation is not restricted or disturbed by the uncoupled development and tests. If a VM breaks down due to a software error, productive operation can continue. That might not be the case without virtualization. This saves costs for personnel as well as time for planning and development.
Scalability and redundancy
A significant advantage of using virtual objects like VMs and containers is that we can duplicate them. If we need multiple servers for heavy load, we do not have to set them up individually with all the software we need, just installing the virtualization software is enough. The VM with the program is then cloned and started on each host. The process, also called clustering, can also take place fully automatically. In this way, redundancies can also be built up and LB can be used.
To go even further, even create a dynamic infrastructure. So we can adjust the number of servers depending on the load, so that fewer servers are online at night than during the day. The user also gets nothing of this, because the load balancer splits the traffic to the different servers. In addition, you can spawn the servers in a data center where the load is generated at the moment. For example, a web hosting may offer geographic distribution to provide as many users as possible with low ping servers. There are now several hosting services that specialize in this type of hosting.
If several servers are used, “rolling restarts” can be realized. All servers are restarted in a certain interval. This process can also be fully automated and, thanks to load balancing, has no impact on the availability of services to the user.
Portability
It worth taking a look at the portability of VMs, where the complex infrastructure in large projects has already been pointed out several times. Portability means delivering all the necessary components to allow a VM to run on as many systems as long as the necessary virtualization software is installed on the host. This property allows you to move VMs to other hosts. If changes are necessary, for example because the network or database configuration needs to be adjusted, these changes are much smaller, easier and faster compared to setting up the software on a new system.
But not only virtual computers, but entire networks can be reconstructed. So you can build a test network, which is very similar to the live network, and there integrate further monitoring and development functionalities. This makes it easy to identify and troubleshoot network issues that might be difficult to find under user load. Likewise, it is possible to intentionally cause problems such as disconnects, packet loss or deliberate overload in order to test and improve the behavior of the applications under difficult conditions.
Automated management
Automation is a very important part of virtualization, as we saw in the Deployment Chain part. This way, all the steps required for continuous delivery can be automated. Releases happen by the way and if you take small steps, you do not have a high risk of failing. But relying on technology and not watching new versions go live, we need a good deployment chain and a good QA and development team.
Likewise, the error handling can be automated. If problems arise, old versions can be restored. In order to go even further, the monitoring processes themselves can run in virtual machines which, among other things, also monitor each other, so that this single point of failure is also ruled out.
Time-to-Market
The time-to-market can be significantly reduced. Sometimes even too few hours between customer requirement and live, without losing quality. But for this to work, it is important that continuous delivery has been well implemented. In order to achieve reliable results, several releases will be carried out on the productive system per day. If an error occurs and significantly interferes with the operation, the changes are reversed at lightning speed. If the bug does not pose a major problem, it will simply be repaired with the next release. All steps are automated. Once the green light is on, all configurations are adjusted, integration tests are performed, and the code is finally updated on all servers or the VMs are exchanged.
Particularly good examples is Google. Google selects random developers to judge new pieces of code. Each tester writes comments, which are then sent to all other testers. After agreeing, the code published or revised depending on the outcome. The testers are not necessarily involved in the project.
Safety and Risk
With virtualized systems you can increase your own security as well as endanger it. Basically, virtualization also helps increase security by separating independent systems so they can not reach each other and share resources. If that one VM gets compromised, it will not affect the other VMs as well as the host. In addition, the virtualization software can restrict access to required systems so that, for example, the web server can reach only the database, not the backup or payment server.
One case that is possible is that an attacker has full access to a VM because he could penetrate a vulnerability of the software used. However, if the attacker finds out that his own server is a VM, he can try to attack the virtualization software. From there, the worst-case access to the other VMs on the same host expands. It is also conceivable that other hosts will be attacked in the same way if they use the same software and are therefore vulnerable to the same vulnerabilities.
If all virtual machines are hosted themselves, the developers or admins can intervene directly and repair the bug on all VMs together. If necessary, service restrictions or downtimes must be accepted. However, this is not possible if you have the VMs with a hoster, because you know there neither the other VMs on its hosts, nor what their job is. In addition, vendors can not shut down and clean up or even rebuild the entire hosts, because VM data may be lost and subject to SLAs.
Similarly, one should, as far as possible, distribute servers of different priorities to different hosts. For example, servers that are publicly accessible should not be connected to internal systems that process credit card or other payment information.
The size of the threat also depends on the form of virtualization used. Hardware virtualization makes it relatively hard to break out. Because you have to get access to the system first, then break any necessary permissions in the guest operating system and ultimately out of the VM. If container virtualization is used, the path is much easier for the attacker. There must only be penetrated into the kernel. Additional layers of protection do not exist in this form of virtualization.
Another point of attack are (especially network-based) backup solutions. If an attacker succeeds in stealing the keys or passwords for the backup server, he may not only have access to the backups of the already adopted system, but also to the others. Possibly the payment server secures there too. Thus, the attacker would not have access to the other systems, but still on the credit cards and the like. Maybe the backup server also stores the login data to the other servers.
Also, the whole environment can become more heterogeneous through the use of virtualization, which can complicate the successful implementation of security concepts.
Virtual Machine and Hardware Virtualization Based Rootkits
With the spread and progress of virtual technologies, the malware is also advancing. So there are so-called stealth malware, or even Virtual Machine Based Rootkits. This creates a virtual machine, which then puts on its own system. Thus, the malware has disappeared outside the user-controlled and monitored area. If the user now scans the system no malware will be found. Thus, the malware is not, or can be found only indirectly, and can spawn new (invisible) virtual machines at will, which can then be remotely controlled and disappear in the end without a trace. However, if the current system is already a virtual machine, this type of malware can lead to problems that can be well understood.
The second form of attack on deep levels of computers is the Hardware Virtualization Based Malware. An example of this is Blue-Pill, which attacks Windows on AMD processors. The procedure is similar. First the program will activate the virtualization function of the processor and then put on the current system, for which now in the processor the guest mode and thus the complete monitoring is activated. Because of this approach, the attack is only active until the next reboot, but the program could write to the autostart section of the operating system to continue to control the computer.
Since there is no real virtualization software with all its features such as virtual hardware used, we can hardly detect the malware. In addition, forensic work is made more difficult. The only protection is to disable the virtualization features in the BIOS, but this is eliminated by using the host as a virtualization server.
Example and Conclusion
Once an application is portable, distributing it across different hosts is no longer a big task. However, the applications must first work stably in VMs. Netflix used to have its own dedicated servers that had a very stable infrastructure with each other because of their physical proximity. But since Netflix is well worth going to the cloud, they had to adjust their software accordingly. At no time can it be predicted how the cloud will look like in the next moment, and whether it will cause problems. The chosen cloud computing builds on fully automated virtualization, so that the human being, if the system runs once, actually does not need to intervene any further. Applications are split into small modular blocks, which then build on each other and are duplicated to provide redundancy. To test this feature, Netflix used the so-called Chaos Monkeys. After they had moved their infrastructure to the cloud, of course, extensive tests had to be done. What happens if a part fails, and how does the cloud work against it? Of course, if a web server fails, a new one must come, and the old one must be stopped and removed. Netflix recognizes that the best protection against a large and unexpected outage is many small failures to extensively test the functionality of the cloud platform and prepare its programs for this type of failure. The Chaos Monkey running in the case on the Amazon Web Services platform, stops random server instances in the network. Accurate monitoring logs these failures. Should there be major problems, the technicians will be informed. The Chaos Monkey also runs live, but only on weekdays and as long as technicians are present to fix any problems that may arise.
Through this complete series, we possibly delivered an overall idea on the topic. We saw that virtualization and the DevOps idea go hand in hand. Virtualization significantly supports agile DevOps thinking and solves or deepens a variety of problems. In practical operation, the implementation of both can prove very useful. But we have to be careful, otherwise agile development can quickly become a mess, especially if the deployment chain was not implemented properly, or the requirements/tickets were inaccurate.
It is often argued that development takes up most of the time. However, DevOps counteracts this.
We still see a lot of potential in the enforcement of the DevOps idea in the market. By forming solutions, new and small businesses could have better success in the market, which are very difficult at the moment because of their size. Also interesting is the idea of automation for the freelancers in the market. As a result, they often resort to less good techniques on how to edit files directly on the live servers during operation.
The big advantage of the DevOps idea is that we can apply it well in both big and small style. This allows a single developer to locally build his (for example) Docker images and then test them locally. Since the environment is completely sealed off, the costs for an additional test server are also saved. If the tests were successful then the image is simply uploaded and started on the live system. If it comes to problems, it is just as easy to start the old image again, as far as they were tagged locally.
Likewise, the DevOps idea can save administrators a lot of time, as the developers themselves can be informed when a problem arises. They then look at it and send their change to the deployment chain, from where it is viewed by all the necessary parties. In addition, admins can take on better-advised functions when planning the infrastructure of a cluster or the like, since administrators in these areas are often better positioned than developers and often participate in training and conferences in larger companies.
The only problem in the DevOps idea that we can see is that when the parties are not interacting properly, it quickly becomes messy. This also means that supporting systems, such as ticket or planning systems, must be properly used, ideally also for communication with the customer. If one looks in the end in his e-mails for details to which no one remembers the execution failed. In addition, new employees who may not be familiar with the DevOps idea may need to be trained accordingly. Likewise, it is unlikely that any developer knows how to properly use Git as the versioning system, for example, so that it can work with full efficiency.
