Posted: April 1st, 2010 | Author: charlie | Filed under: IT Management | Tags: linux, ROI, web hosting | 1 Comment »
Whether you are a small business, fortune 500, or in-between, Web hosting providers may have something to offer. Web hosting is extremely competitive, and not many companies survive. Once the need for Web hosting services is established, you need to do some research to determine whether a specific hosting company is a good fit.
Small businesses will depend more heavily on their hosting provider, as their entire Web infrastructure may reside at a hosting company. As businesses get larger, and acquire their own IT staff, they tend to start running their own Web servers. At some point, maybe due to cost-cutting measures, companies may revisit the hosting option. If it’s really only $5 per month, then why not?
Hosting companies also offer virtual machines that your IT staff can run. These are a great alternative to hosting DR servers in remote data centers. VM plans are more expensive than a regular shared Web hosting plan, but not much more. With both options we need to pay close attention to the hosting company’s track record and value-add services.
What to Expect
There are literally hundreds of popular hosting companies out there. Each has their own tools for managing your domains, and each offers their own unique set of value-add services.
At a minimum, there are a few things you should demand from any Web host. Due to the brutally competitive nature of the business and interchangeability of these services, you should never be shy about demanding a new feature or picking up and moving to a new provider. A few of the basic things you should expect are:
- A Web interface to manage your domains, billing, and user accounts
- Shell, SFTP, and various other methods of remote access
- Unlimited MySQL databses, and an easy way to create and manage them
- Easy, automated installation mechanisms for common open source Web applications
- Unlimited e-mail boxes and a few options for Web-based access
- Log analysis and detailed reporting
- And lately: unlimited disk space and bandwidth
Five or ten years ago you would not find most of these features with the most popular hosting companies. These days, however, they all offer most of these items and a lot more.
Companies that offer VM hosting, or Virtual Private Servers (VPS) as some call it, will provide a basic set of tools to manage your virtual machine. You can generally select from a few operating systems and reload it at any time. Afterwards, however, you’re on your own. You get the root password and are free to install whatever you need.
Is it Worth It?
Psychologically, most people have a problem purchasing something that appears too cheap. When looking at Web hosting services, the list of options leaves most people drooling. Then they see the price: sometimes less than $5 per month, for unlimited disk space, bandwidth, accounts, and with a free domain registration. How can they do that?
The servers must be horribly overloaded as these companies cram more and more customers onto them. At $60 per year per customer, the hosting company can’t exactly afford to buy top of the line servers, right?
The truth is, hosting is brutally competitive. Most companies fail. The ones that survive have a high level of automation to allow them to manage their servers with very little manhours. Even at today’s sub-$5 pricing for a monthly plan, that equates to one $3000 server per 50 customers, at their yearly rate. A properly configured Apache Web server environment should be able to handle 300 Web sites on today’s hardware without a problem.
Of course, someone could write a PHP script that goes wild and consumes way too many resources, slowing down the entire server. The hosting companies monitor the servers, and this type of thing is usually dealt with quickly. However, it is worthwhile to ask your chosen hosting company whether or not they allow adult content. Adult sites are generally very high-traffic, and if you’re site is sharing the same Web server, it may suffer performance issues.
The Case for Secondary Hosting
Enterprises should not rule out hosting providers. You don’t need to outsource your entire Web team, but part of a good disaster recovery strategy will certainly include multiple off-site resources. Instead of paying co-location fees and trodding your own servers to a datacenter across the country, why not get a few virtual machines in multiple locations?
If your physical server dies, you may be sending FTE to the remote site to replace the hardware. In a hosted VM environment, you never have to worry about taking employees away from their daily work and paying travel expenses. Just click a button in the hosting provider’s Web interface to create a new VM if something goes awry.
You can host your own Web servers, secondary DNS, or just about any other service your company relies upon. If you’re worried about reliability, you can always select two or three VM providers in separate parts of the country (or world). To equal the cost of one server in a co-location environment, you’d have probably 10-20 virtual machines spread throughout the world.
Next Steps
If you’re interested in hosting Web sites with a Web hosting provider or VMs, the next step is to do some research. Read reviews of various hosting companies, and pay attention to issues people cite. There will always be a certain amount of people unhappy with a service, but pay careful attention to what they are saying went wrong. Likewise, pay attention to the few positive reviews you might find. Most happy customers remain silent, so when you find a positive review it holds more weight than a rant.
“Cheap web hosting” is a great Google search. You will find a few unbiased sites that list providers, the services they offer, reviews, and pricing. We don’t want to make any recommendations, but just remember: more expensive is not always better in a fiercely competitive market such as Web hosting.
1 Comment » Related posts:
- Managing Virtual Machine and Cloud Sprawl
- How Much Server do you Need?
- Zenoss: We Can Ditch Nagios Now
- Understanding Linux Virtual Memory
- The Perils of Sudo With User Passwords
Posted: March 1st, 2010 | Author: charlie | Filed under: IT Management | Tags: capacity, performance, tuning, virtualization | 3 Comments »
When purchasing server hardware, do you tend to purchase more power than you need, or not enough? Specifying the correct server for your current need is a fine art, and it’s easy to get wrong. Here are some helpful hints and considerations to remember that will ensure you make the right server purchasing decision.
We’re going to focus on standalone (non-blade) servers for the moment, but many aspects are also applicable to blade servers. Blade servers are wonderful for centralized management of the hardware, but the specs of the individual server blades can vary tremendously.
Hardware Management
Want to avoid trudging down to the datacenter late at night, or even worse, across the world if something breaks? Then don’t skimp on the management controller, lights out manager, or whatever the vendor is calling it. Many vendors ship a simple version by default: it may allow serial console access only, for example. Make sure to get the full-featured controller, because even if the hardware is only a few doors down, getting up from your desk should never be necessary.
If you aren’t thinking of switching vendors any time soon, you might think that the management interface will always work the same as it has on all your other servers. Unfortunately, that’s not the case. Sun x86 hardware, for example, has many different hardware management controllers to choose from. The more expensive and feature-rich servers have the better controllers, but don’t make the mistake of thinking the interface never changes. The unfortunate part is that you never know how well it works until you get a server on-site.
Hardware management comes in two forms: IPMI (most support), and the user interface. The user interface is more often than not, a Web-based java application that provides remote console access. Some are extremely buggy, and others work quite well from all Web browsers. We can’t make a recommendation, though, because these things change often.
Memory
Shucks, this one is a no-brainer: as much as you can afford. Within reason, that is. If you aren’t going to run virtual machines, and this server’s only job is to serve up some simple Web pages, then 16GB of RAM is likely overkill. Likewise, make sure you know what your application can support. Many java applications are limited to a heap size of 2 or 4GB.
It’s also overkill to purchase more than 4GB of RAM if you need to run a 32-bit operating system. Yes, Windows Server does some tricks and it can use more than 4GB, but it’s a huge performance it.
If virtualization is in your future, load up as much as possible. You also want to pay attention to how many DIMM slots the server has. The 8GB DIMMs are horribly expensive now, so you’ll probably want to stick with 4GB sticks. Just remember, if you fill all the slots in the server, the only memory upgrade path is to buy higher capacity DIMMs.
CPU
Do you want to run many threads at an even pace or just a few threads as fast as possible? Sun’s T2 processors aren’t fast by any measure, but they can run many threads at the same speeds consistently. These are ideal for database servers, but not for Web servers.
Will this server be executing a wide variety of processes over and over again, as opposed to just running the same big application server constantly? If so, make sure you pay attention to the amount of cache each core of the CPU has.
For virtualization, you want the fastest multi-core processors available, with the largest amount of L2 cache. Cache is very important as it minimizes the number of times the CPU needs to fetch data from slower RAM. It makes a very noticeable difference on heavily used servers.
Disks, Controllers, and RAID
If you need local storage, do pay attention to the type of disks you’re ordering. A SATA disk is likely to disappoint if you have an IO-heavy workload. SAS, and FC disks should perform equally well, since they are both SCSI disks underneath.
Even if you don’t need much local storage, you should always buy a server with a RAID controller that can mirror the operating system disks, unless you’re SAN booting of course. You don’t want the OS to crash just because of a failed disk. Likewise, if you’re keeping tons of local storage for some reason, make sure to get a RAID card that does RAID-5, so that you can at least lose one disk at a time without losing data. If performance is a concern you should really be using iSCSI or SAN storage, but you may also think about a RAID 0+1 configuration to avoid the slower RAID-5 parity calculations.
If you’re attaching to a SAN, make sure to include the correct HBA as well.
Networking
When servers started showing up with two or four gigabit NICs I must admit, I was confused. Why would someone need that many? Aside from large servers that do a lot of network IO, you might also want to separate out your iSCSI traffic from normal Ethernet. It’s also important these days to make sure that the network cards support TOE, or a TCP Offload Engine. This will task the network card with computing TCP checksums, freeing your CPUs for more important things.
In summary, most of these things may seem common sense, but you need to remember to ask all the right questions every time you spec a server. Here’s a good checklist:
- Adequate hardware management controller
- Enough (but not too much) RAM, that’s fast enough, but not faster than the CPU’s front-side bus
- Enough memory slots for expansion, if that seems likely
- Correct CPU for this server’s needs
- RAID-1 for the OS, and (optionally) other RAID levels for other local storage
- FC HBAs?
- Multiple gigabit NICs with TOE capabilities
3 Comments » Related posts:
- Managing Virtual Machine and Cloud Sprawl
- SAN 101: Intro to SANs and Storage
- Understanding Linux Virtual Memory
- Back to Basics: Unix System Stats Utilities
- Squeeze Your Gigabit NIC for Top Performance
Posted: February 25th, 2010 | Author: charlie | Filed under: IT Management, Linux / Unix | Tags: linux, Security | 28 Comments »
The consensus among new Unix and Linux users seems to be that sudo is more secure than using the root account, because it requires you type your password to perform potentially harmful actions. In reality, a compromised user account, which is no big deal normally, is instantly root in most setups. This sudo thinking is flawed, but sudo is actually useful for what’s it was designed for.
The (wrong) idea is that you shouldn’t use the root account, because apparently it’s too “dangerous.” This argument usually comes from new Linux users and people that call themselves “network administrators,” but has no basis in reality. We’ll come back to that in a moment.
The concept behind sudo is to give non-root users access to perform specific tasks without giving away the root password. It can also be used to log activity, if desired. Role-based access control isn’t available in Linux, so sudo is a great alternative, if used properly. Solaris 10 has greatly improved RBAC capabilities; so you can easily allow a junior admin access to web server restart scripts with the appropriate access levels, for example. Sudo is supposed to be configured to allow a certain set of people to run a very limited set of commands, as a different user.
Unfortunately, sysadmins and home users alike have begun using sudo for everything. Instead of running ‘su’ and becoming root, they believe that ‘sudo’ plus ‘command’ is a better alternative. Most of the time, sysadmins with full sudo access just end up running ‘sudo bash’ and doing all their work from that root shell. This is a problem.
Using a user account password to get a root shell is a bad idea.
Why is there a separate root account anyway? It isn’t to simply protect you from your own mistakes. If all sysadmins just become root using their user password (running: sudo bash), then why not just give them uid 0 (aka root) and be done with it? For a group of sysadmins, the only reason they should want to use sudo is for logging of commands. Unfortunately, this provides zero additional security or auditing, because an attacker would just run a shell. If sysadmins are un-trusted such that they need to be audited, they shouldn’t have root access in the first place.
Surprisingly, the home-user rational makes its way into the workplace as well. The recurring argument is that running a root shell is dangerous. Partially to blame for this grave misunderstanding is X login managers, for allowing the root user to login. New users are always scolded and explained to that running X as root is wrong. The same goes for many other applications, too. As time progressed, people started remembering that “running as root” is wrong, passing this idology down to their children, but without any details. A genetic mutation may have occurred, but insufficient research has been done on that topic thus far. Now that Ubuntu Linux doesn’t enable a root account by default, but instead allows full root access to the user via sudo, the world will never be the same.
People praise sudo, while demeaning Windows at the same time for not having any separation of privileges by default. The answer to security clearly is a multi-user system with privilege separation, but sudo blurs these lines in its most common usage. The Ubuntu usage of sudo simply provides a hoop to jump through, requiring users to type their password more often than they’d like. Of course this will prevent a user’s web browser from running something as root, but it isn’t security.
We’d really like to focus on the Enterprise, where sudo has very little place.
The sudo purists, or sudoists, we’ll call them, would have you run sudo before every command that requires root. Apparently running ‘sudo vi /etc/resolv.conf’ is supposed to make you remember that you’re root, and prevent mistakes. Sudoists will also say that it protects against “accidentally left open root shells” as well. If there are accidental shells left on computers with public access, well that’s an HR action item.
Sudo atheists will quickly point out that using sudo without specifically defined commands in the configuration file is a security risk. Sudoists user account passwords have root access, so in essence, sudo has un-done all security mechanisms in place. SSH doesn’t allow root to login, but with sudo, a compromised user password removes that restriction.
In a true multi-user environment, every so often a root compromise will happen. If users can login, they can eventually become root, and that’s just a fact of life. The first thing any old-school cracker installs is a hacked SSH program, to log user passwords. Ideally, this single hacked machine doesn’t have any sort of trust relationship with other computers, because users are allowed access. The next time an administrator logs into the hacked machine, his user account is compromised. Generally this isn’t a big deal, but with sudo, this means a complete root compromise, probably for all machines. Of course SSH keys can help, as will requiring separate passwords for administrators on the more important (non user accessible) servers; but if they’re willing to allow their user account access to unrestricted root-level commands, then it’s unlikely that there’s any other security in place elsewhere.
As we mentioned, sudo has its place. Allowing a single command to be run with elevated privileges in an operating system that doesn’t support such things is quite useful. Still, be very careful about who gets this access, even for one item. As with all software, sudo isn’t without bugs.
For the love of security, please, we beg of you, do not use sudo for full root access. Administrators keep separate, non-UID 0 accounts for a reason, and it’s not for “limiting the mistakes.” Everything should be done from a root shell, and you should have to know an uber-secret root password to access anything as root.
28 Comments » Related posts:
- Multi-user Security in Linux
- Back to Basics: Unix File Permissions
- Working With Unix Variant Differences
- Managing Virtual Machine and Cloud Sprawl
- Back To Basics: Unix Differences in Performing Tasks
Posted: February 15th, 2010 | Author: charlie | Filed under: IT Management, Linux / Unix | Tags: configuration management, linux, virtualization | No Comments »
Virtualization (in the cloud or locally) is great; that much we can all agree on. Virtual machines (VMs) can tend to grow out of control, however, now that it’s so easy to create them. This should not be all that surprising, but many small to medium businesses are also dabbling in VMs, and they are suddenly overwhelmed by the VM growth.
Each VM is another server that an administrator must manage. Security updates must be applied and global configuration changes now need to be propagated to all these new machines. While it’s easy to create 3-4 (or more) servers on one physical piece of hardware, you’ll certainly struggle if you aren’t already set up to scale.
Unfettered Growth
The number of physical machines in a small company may drop dramatically; maybe 40%, when virtualization is implemented. Unfortunately, the number of OS instances will generally increase by two-fold or more at the same time. The power and cooling savings are realized, as was promised by virtualization, but taking 20 servers to 12 servers, for example, will means you may soon have 40 OS instances to manage.

Puppet, from Reductive Labs
The reasons for VM proliferation depend on your culture, but the most common reason is that delegating control of an entire OS is easier than managing an application for customers. IT customer, be they engineers, application developers, or smaller IT units within an organization, frequently need more access then cenral IT is willing to give. The easy solution: give them a server of their own. Test environments, too, are best served by virtual machines.
To keep hardware (and power and cooling) costs down, many companies implement policies about the implementation of new services. New applications and servers need to be run on VMs first, unless it’s really requires its own server. Policies such as these are good, in that they limit wastefulness, but they do tend to exacerbate VM sprawl.
Sprawl aside; it’s worth noting that higher utilization levels on your servers does not mean that they’ll use an appreciably larger amount of power. In fact, the power savings claims are really true, and can be even greater if your utilization is low and you use VirtualCenter’s power management features. VMWare can migrate VMs to fewer servers if utilization isn’t high enough, and actually power off unnecessary servers. This works best with Dell hardware, but other large vendors are supported as well. Imagine: all your VMs migrating to a few blades in a blade server during the nighttime, and then as utilization increases during the day, blades quickly boot up and take the load as needed. Granted, I don’t personally know any enterprise environments that are brave enough to try it yet, but in theory the concept is wonderful.
Dealing
Something magical happens when a company grows to around 50 operating systems. It’s too many to manage by simply logging in and running commands, so people start to write scripts. In Windows land, if it hasn’t already happened, you must implement Active Directory. For the Unix/Linux servers, configuration management becomes even more important. Writing a script that SSH’s to each server and runs a command doesn’t scale, no matter how hard people want it to. You need a real configuration management system (such as puppet or cfengine) to ensure that servers are configured exactly how you want, and that they will remain that way.
If you already operate in a large environment with good automated installations and configuration management systems, chances are scaling 100-fold won’t be a problem. Barring scaling issues with the management software itseld, that is. A good network-booting deployment system is only half the battle, because every server isn’t going to be configured identically. If you’re “doing it right,” you should be able to arbitrarily reinstall any server, walk away, and know that it’ll come back up patched and running all the services it’s supposed to. Servers, or rather the OS that runs on them, should be truly disposable.
Management of a “golden image” is promised by VMWare, probably because ITIL mentions it, but it doesn’t really help in practice. You have to create your images (somehow). There’s no mechanism to update a golden image with security patches and apply them to existing systems; you’ll generally have to reinstall the OS instances. And that’s what you should do periodically, but without some kind of configuration management system, you’ll also be manually installing and configuring the services that the VMs used to provide in order to restore service functionality.
VM growth, therefore, is no different from server growth. It may be easier and cheaper, but from the OS management viewpoint, you’re doing the same thing. Likewise, the availability of your services is also in danger. Running five VMs on a single piece of hardware means that a hardware failure takes out five servers instead of one. VMWare and Xen can both be clustered and run from shared storage, such that a hardware failure will result in the VMs immediately (instantly, even) being migrated to other servers. The problem is that VMotion requires the most expensive VMWare license, and a VirtualCenter server. Shared storage isn’t as big of an issues these days with iSCSI, but its still another aspect that must be configured. We’ll cover this issue in-depth in a future article, focusing on Xen and RHEL Clustering Services.
The point is: dealing with VM sprawl is no different than dealing with scaling up to support more physical servers. Use whatever mechanisms are available on your given platforms, and “do it right.” A VM is, and always will be, just another server.
No Comments yet... be the first » Related posts:
- Understanding Linux Virtual Memory
- How Much Server do you Need?
- Zenoss: We Can Ditch Nagios Now
- Is Cheap Web Hosting Worth It?
- The Perils of Sudo With User Passwords
Posted: February 14th, 2010 | Author: charlie | Filed under: IT Management, Linux / Unix, Networking | Tags: linux, monitoring, networks, NMS | 16 Comments »
Another perfect example of open source software gone commercial is Zenoss. As a full-featured network and service monitoring solution, Zenoss is one of the best monitoring tools available.
Most importantly, Zenoss combines two functionalities. First and foremost an enterprise environment requires host and service monitoring, with notifications. Network monitoring really means checking services, checking that hosts are up (they ping), and possibly writing your own plugins to check various other aspects of a server or network device. Until now, Nagios has filled that role.
Second, once a decent monitoring solution is in place, getting time-based information becomes desirable. Memory and CPU usage is the most prevalent example: if you’re checking available swap space every so often with Nagios, you may know when you start running low. But it may be just as important to see a graph of the last week’s usage. Tools like Cacti or Munin, which collect data frequently and use RRD graphs to display it, are very useful.
Zenoss fills both roles, without the annoying shortcomings prevalent in the alternative solutions. Zenoss uses the terms Availability Monitoring and Performance Monitoring to describe these two fundamental roles.
Performance of monitoring tools is important, and often times overlooked until it becomes a debilitating problem. For example, if you want to chart pretty RRD graphs of systems statistics like available RAM or disk space, Munin is an option. Unfortunately it’s all Perl, and designed in such a way that prevents it from scaling to even moderate amounts of hosts. Cacti is a bit better, but monitoring close to 100 hosts is painful with either option. Along comes Zenoss.
Zenoss is written in Python, and uses a MySQL backend for storage, and by all accounts it appears to perform very well. The really great thing about corporate-backed open source is quality control. The community simply isn’t responsible enough to say, “No, this won’t work, re-implement it.” A company with QA is.
Speaking of features, Zenoss isn’t missing many. Flexibility seems to be top priority–it can monitor hosts with SNMP, Nagios agents, SSH, Windows WMI, and various other mechanisms. Many features they claim are a bit over-inflated, such as ZenPing (marketed as Network Topology Monitoring) but the feature set is rich nonetheless.
Zenoss’s primary functions involve four features:
- Inventory Tracking
- Availability Monitoring
- Performance Monitoring
- Event Monitoring and Management
Inventory tracking claims some sort of “configuration” reporting as well, but it seems very limited. Zenoss will discover your inventory and auto-populate a database. This is great for knowing which IP addresses are in use, for example, but means that “configuration” reporting is limited to an outside observer’s perspective. It can tell you which servers have a Web server running, but it certainly doesn’t deal with the configuration of the Web server. Of course, inventory tracking isn’t limited to automatically discovered information; there are manual input capabilities too.
Availability monitoring is basically Nagios, plus. It can ping, it can monitor Windows machines, and it can pretty much do whatever you need. Even your old Nagios plugins will work with Zenoss. It does generate reports, but much better ones than Nagios is capable of.
Host monitoring, performance monitoring, or whatever you’d like to call it, is quite robust in Zenoss. Some would think it’s light on features, but there’s a good reason that Zenoss requires you use SNMP: it’s much more scalable than SSH’ing to each server every minute. A bit of up-front configuration is required, in that all your hosts will need SNMP configured and working, but it’s completely worth it. Zenoss too uses RRD graphs, and it can generate events and alerts based on pre-defined thresholds.
Finally we come to event monitoring. Zenoss is also encroaching on Splunk‘s territory a bit. It can combine syslog, availability monitoring alerts, SNMP traps, and even Windows event log data. Much like Splunk, Zenoss correlates similar events for easier viewing and troubleshooting. This is the portion that processes all events and generates alerts to pagers or e-mail, taking into account the escalation procedure you’ve defined.
To top it all off, the Zenoss Web interface is top-notch. It includes a customizable “dashboard” for monitoring, and everything is AJAX-enabled. AJAX provides the user experience similar to Splunk and Google’s Gmail.
Marketing fluff aside, Zenoss really does provide a wonderful product. It is, of course, open source and available for free.
At last year’s LISA conference, Zenoss gave a demonstration that sadly coincided with free beer time. Stumbling in toward the end, I demanded one of their free baseball caps, and sat to listen to the last few audience questions. One thing was very obvious: everyone in the room was excited about this product. If hardcore sysadmins are excited, you know this is something worthwhile.
Zenosss is very functional and full of features. It may even be possible to replace three separate pieces of software with this one product: host inventory database, Nagios, and your performance monitoring tool of choice. Maybe even Splunk some day. We can’t wait to see what features they will be adding next.
16 Comments » Related posts:
- Squeeze Your Gigabit NIC for Top Performance
- Managing Virtual Machine and Cloud Sprawl
- Built-in Security with Cisco IPS
- Back to Basics: Unix System Stats Utilities
- Manage Devices and Configurations with Cisco SDM