Monitor Your Website from a Shell Script

My organization currently runs a homegrown shell script solution. Although I built it and maintain it, I realize it has limitations.

My Homegrown Website Monitoring Solution

An extended outage in November 2002 spurred me to implement external monitoring of my organization's web servers. Someone misconfigured our external DNS settings and knocked all our web and mail servers off the Internet for two to three days. However, our internal DNS contained the correct settings, so we did not notice the problem until users started calling.

External monitoring is important because it warns us early if users outside our network cannot see our web sites. My department had no budget for this, so we implemented a shell script running on my personal hosting account on a shared FreeBSD server. This system is still in use.

Every half hour, a shell script runs on my personal account. This downloads the HTML source code from a list of URLs and searches for particular keywords within that code.

If the script does not find the desired keyword at a particular URL, it sends an e-mail to a URL-specific list of users. Some of the e-mail addresses point to staff members' pagers or cell phones. Regardless of success or failure, the script logs the results, but these are sitting in a personal account and not readily accessible for running statistics.

Resources

The script is based on the source code in the article below from ;login: The Magazine of USENIX and SAGE. I want to thank the staff of ;login: for providing me a copy of this issue so many years ago. I was not a subscriber, but I asked nicely on behalf of my non-profit, and they sent the issue right along.

http://www.usenix.org/publications/login/

Monitoring Strategies for High-Availability Web Apps by Neil Ashizawa
http://www.usenix.org/publications/login/2002-08/

Comments

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.