You may not have, as I did, the luxury of choosing the array of hardware – rack-mount servers (RMS) or Blade servers - to be used for your OPNFV implementation. I’m going to assume you’re not rolling in a rack full of blades, fiber switch fabric, and storage arrays, but have to make do with what you hope are adequate 1U or 2U machines. I’ll mention later some of the interesting twists encountered with a blade setup, but let’s start with RMS’s in the blog.
First, how many do you really need for a minimal but usable deployment? Yes, it is possible to do everything on a single large system by putting the various OpenStack nodes on virtual machines (often Oracle VBox). But, this leads to extra network configuration as what are normally physical NICs will now be virtual NICs. And, worse yet, the machine that will run your guest VMs will itself be a virtual machine. This added layer of virtualization will not be a pleasant thing.
In my mind, a set of 4 bare metal systems is needed for a minimal OPNFV deployment. This will give you a Fuel master node to deploy from and a place to park all of the necessary roles required for OPNFV. What you will lose is some speed because you have jammed several functions on the same system, and you will sacrifice redundancy/high availability. In addition, this configuration only provides a single compute node. So, this node should at least be a sizable system. My compute has 8 CPUs and 32 GB of memory – enough to start up at least a handful of guest VMs.
Let’s look at the overall picture of the deployment. Note that this shows only the RMS side of things, although the virtual Fuel machine is there.
Here are some important points:
- Machines need a minimum of 2 NICs. The NICs (or at least one NIC) must support VLANs.
- 1000BT NICs are the absolute minimum. 10 GB/s NICs would not be overkill! Our blade server deployment uses 10 GB/s and is noticeably faster.
- All internal OpenStack networks coexist on a single physical network by means of VLANs. The external/public network is also on a VLAN.
- The PXE boot network must be a flat (non-VLAN) network.
- Gateways in the ToR switch route traffic to other machines in the development lab environment, general corporate network, and public internet.
- Corporate and lab DHCP must be segregated from OPNFV/Fuel/OpenStack. DHCP servers are automatically set up there as needed. If there are conflicting DHCP servers, you will be in trouble.
If you are not huddled next to your stack of systems, you will likely wish you had viable remote consoles. Unless remote management capabilities are built into the server, this would be through an Ethernet-enabled KVM switch. You will probably need to change BIOS settings, check the progress of PXE boots and get to the systems when networked ssh is not functioning for reasons unknown.
Let’s move on to the Fuel setup. As a quick refresher, Fuel is an installer for your OpenStack/OPNFV environment that provides a GUI to lead you through the OpenStack configuration (If you need a refresher, here’s the link to an earlier blog). As I have mentioned, the OPNFV Fuel Installation Guide is quite comprehensive, laid out in cookbook fashion so that it can be followed from start to finish. So, nothing will be gained by rehashing it. What I will do is point out some “gotchas” that got me in the hope that they won’t get you.
Fuel setup tips:
- FUEL menu is kind of fidgety, especially with a remote console. Make sure the console is big enough, or strange behavior will occur. Status/error line can be lost, labels truncated and tabbing between fields may not work correctly. Make sure you have a full 80 character width and that you see the full menu:
- Most screens in Fuel have a “verification” function. Always give this a try before saving the values for the screen.
- Remember that there is a running system behind the Fuel menu. You can drop into it to verify things on the underlying system (most importantly, network connectivity) using “Shell Login.”
- The Fuel menu can be run after the installation to easily check on Fuel settings – “fuelmenu” will start it up. Remember to Quit Setup. Quit without saving so that it will not try and make any changes.
- I had trouble getting to the outside world for Ubuntu and OpenStack repositories and thus passing network verification. I set “Skip building bootstrap image” in the Bootstrap Image screen and set up the repositories later, before deploying.
- While it’s possible to set the default gateway on the screen for either network, you will likely only set it as part of the non-PXE network.
- Fuel on a virtual machine – worth the effort if you are using multiple deployments. While multiple environments within a single Fuel are possible, separate Fuels are more flexible, especially in a “mixed” (Blade and RMS) environment. “Bridged” networking devices should be used, as they explicitly map a VM’s virtual network device to a physical device on the host. This is important, as one of the physical networks is flat and used for PXE booting and the other is divided up into VLANs. Also, check closely to make sure the physical device you expect is connected to the bridge device you expect. Ethernet naming conventions can be confusing.
- Be patient once you start the Fuel setup. It can take a while. When finished - if you can get to the Fuel GUI - all is well.
- When booting the OpenStack nodes to make them available for Fuel, it is helpful to watch what happens on the console. If your PXE boot is not working, you may need to drop back into “legacy” boot mode instead of UEFI. I found I had to do that. This will depend on your servers. When the PXE boot works properly, OpenStack nodes will magically appear as ready-to-deploy in Fuel.
- If “Skip building bootstrap image” was chosen, you will receive a suggestion to set up a local repository on the newly-installed fuel node, using “fuel-createmirror.” Follow this suggestion, as this will replace the repositories you were not able to install as part of the Fuel node installation. This will conveniently update all of the URLs in Settings/General/Repositories screen.
As with the Fuel installation, the OPNFV Fuel Installation Guide does a good job of laying out the steps of setting up for and doing the OpenStack deployment using the Fuel GUI. However, there are some traps you can possibly fall into along the way. My next blog entry on my OPNFV journey continues with exposing some of those traps as well as highlighting some additional points to think about in deploying your OPNFV platform.
This is part 4 of the "OPNFV Demystified" blog series. The next part of the blog series will be posted next Thursday morning.
Check out the other posts in this series.
- Intro to Building a Carrier Grade Cloud Operating System Using OPNFV (OPNFV Demystified Part 1)
- The Components of OPNFV (OPNFV Demystified Part 2)
- Getting Started with OPNFV - How Should I Prepare? (OPNFV Demystified Part 3)
- Real-World OPNFV Implementation – The Hardware (OPNFV Demystified Part 4)
- Real-World OPNFV Implementation – Traps and Tips (OPNFV Demystified Part 5)
- Real-World OPNFV Implementation – Networking (OPNFV Demystified Part 6)
- Why Automation Is Key to Your OPNFV Deployment (OPNFV Demystified Part 7)